Question: In a few sentences, explain why manually reviewing about 100 examples that the algorithm performs poorly on can help identify major categories of training set errors, and how this relates to the dev set error analysis process.

Sample answer: Manually reviewing about 100 poorly-performing training examples allows a practitioner to directly observe the specific ways the algorithm is failing, rather than relying only on aggregate error metrics. By counting these errors into categories, one can identify which types of mistakes are most common, which mirrors the dev set error analysis process of counting errors across categories to prioritize fixes.

Key points:

• Reviewing ~100 poorly-performing examples
• Goal is understanding major categories of training set errors
• Errors are counted across categories
• Process parallels dev set error analysis

Rubric: Full credit requires mentioning: (1) the practice of reviewing ~100 examples, (2) the goal of identifying major error categories, (3) the connection to counting errors similar to dev set analysis.