Question: Explain, using the concept from the source, why progress on a machine learning problem tends to slow down once the algorithm surpasses human-level performance, and why the opposite is true while machines are still catching up.

Sample answer: While an algorithm is still below human-level performance, humans can easily spot examples where it clearly errs, enabling techniques like error analysis and human comparison to drive rapid improvement. Once the algorithm surpasses human performance, humans have a hard time identifying its remaining mistakes, so only a subset of human-comparison techniques still apply, which slows the rate of further progress.

Key points:

• Progress is faster while machines are still catching up to humans
• Progress is slower once machines surpass human-level performance
• Humans have a hard time identifying clearly wrong examples once performance surpasses human level
• Only a subset of human-comparison techniques still apply in that case

Rubric: Full credit requires explaining both the catching-up and surpassing cases, referencing the loss of human ability to spot clear errors, and connecting this to the reduced set of applicable techniques.