Question: Describe why the share of total mistakes coming from one category sets an upper limit on how much the overall error can improve if that category is fixed perfectly. Explain why this is useful for choosing which machine learning work to do first, and include a concrete example.

Sample answer: If a category accounts for only a small portion of all errors, then even eliminating every mistake in that category can only reduce the total error by that small portion. That share is therefore an upper bound on the gain from perfecting that category. This matters for prioritization because teams should spend effort where the largest total improvement is possible, not where the most convenient fix exists. For example, suppose a classifier makes mistakes on 200 out of 2,000 cases, so the error rate is 10%. If 20 of those mistakes are in one category, then fixing that category perfectly can lower the overall error by at most 1 percentage point, from 10% to 9%. If another category accounts for 100 mistakes, fixing it perfectly could reduce the error by 5 percentage points, which is far more valuable.

Key points:

• Defines a category’s error share as the maximum possible reduction in total error from fixing that category.
• Explains why this ceiling helps prioritize work with the largest expected impact.
• Gives a numerical example showing that a small error share yields limited total improvement, while a large share yields a bigger payoff.

Rubric: The response should define the category’s error share as an upper bound on total error reduction, explain its role in prioritizing machine learning work, and include a clear numerical or conceptual example demonstrating the limit.