Case context: Your speech recognition team has significantly improved their model, reducing the overall error rate to just 2%. The team lead wants to conduct a manual error analysis to find new areas for improvement and asks you to assemble an Eyeball dev set that contains approximately 100 misclassified audio clips.

Question: Calculate the total number of audio clips you need to include in the Eyeball dev set to meet the team lead's requirement, and explain the mathematical reasoning behind your decision.

Sample answer: You need to include approximately 5,000 audio clips in the Eyeball dev set. Because the classifier's error rate is 2% (0.02), you divide the target number of errors (100) by the error rate to find the total set size needed. This demonstrates that a lower error rate requires a significantly larger Eyeball dev set to accumulate enough errors to analyze.

Key points:

• Calculate total size by dividing target errors by error rate (100 / 0.02 = 5000)
• Recognize the target of ~100 misclassified examples for manual review
• Explain that lower error rates demand larger Eyeball dev sets

Rubric: Award full credit if the learner correctly calculates 5,000 examples and explains that dividing the target number of misclassified examples (100) by the low 2% error rate necessitates a large dataset.