Question: Explain how measuring the frequency of different error categories in a misclassified development set helps a machine learning team decide what to improve first. Give a specific example.

Sample answer: Counting error categories tells the team how much benefit a fix could possibly provide. For example, suppose an audit of 120 development-set mistakes shows that 12 come from poor nighttime visibility, while 36 come from missed stop signs. Even if the team solved every nighttime-visibility case, it could reduce total errors by only 10%. In contrast, addressing missed stop signs could affect up to 30% of the errors, so that category deserves higher priority. The key idea is to focus first on the error types that occur most often, because they offer the largest potential payoff.

Key points:

• Error frequency sets an upper limit on the improvement a single fix can deliver.
• A category that appears in only 12 of 120 errors has a smaller payoff ceiling than one that appears in 36 of 120.
• Categories with the highest counts should usually be prioritized first because they can reduce more total mistakes.

Rubric: The essay should explain that frequency bounds the potential impact of a fix and correctly use the example to show why the more common error category should be addressed before the less common one.