Upper bound on error reduction from one source
Question: A document classifier makes 15% of its current mistakes on pages with faint print. Why does that 15% act as a ceiling on the improvement you can get by fully fixing faint-print handling?
Sample answer: It is a ceiling because fixing faint print can remove only the mistakes caused by that issue. If faint print accounts for 15% of the current errors, then even perfect performance on that case can reduce the total error by at most 15% of the current error count. Errors caused by other issues would still remain.
Key points:
- The fraction for that category is the largest possible reduction from fixing it.
- Improving one category cannot eliminate errors that come from unrelated causes.
Rubric: The answer should clearly state that the category's error fraction is the maximum possible reduction in overall errors achievable by fixing that category.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What does an error-category fraction mean as an upper limit in error analysis?
If 8% of the mistakes in a classifier come from low-light photos, then making low-light detection better will always eliminate 8% of all errors.
Category error share as a limit on improvement
Match each share of category errors to the best possible reduction in a system with an 8% error rate.
Use category fractions to judge whether an error type deserves attention
A classifier has 20% error, and 30% of its mistakes are misclassified bicycles. What is the best possible accuracy after perfectly fixing bicycle classification?
If one error category makes up a large fraction of mistakes, then fixing that category will definitely produce a large improvement in total performance.
If a model has a 12% error rate and 8% of those errors come from one specific class, the best possible error rate after fixing that class perfectly is _____.
Match each term from improvement-cap analysis with its meaning.
Order the reasoning steps for using an error-category fraction to judge priority.
Using a category’s error share to estimate its improvement limit
Which defect should a warehouse robot team fix first?
Upper bound on error reduction from one source