Question: Discuss why examining roughly 50 mistakes in a manually inspected validation set is often enough to get a useful picture of the main failure modes. Why can fewer examples be misleading, and why does checking far more than 50 often add limited value for an initial review?

Sample answer: Looking at about 50 mistakes gives enough evidence to notice the main repeating patterns without creating too much manual work. If you inspect only a small handful, a few unusual cases can dominate your impression and hide the more common problems. If you keep going much beyond 50, you usually spend a lot more time but learn only a little more for a first pass, because the dominant error categories are already clear. So 50 is a practical middle ground between reliability and effort.

Key points:

• Around 50 errors is enough to see common patterns.
• Too few examples can overemphasize outliers.
• Much more than 50 often has diminishing returns.
• The goal is a useful first estimate, not a complete audit.

Rubric: Responses should explain both why a small sample can be misleading and why a larger sample quickly becomes less efficient for an initial error review.