Prioritizing Error Categories in a Vision Classifier
Case context: A team has completed an error analysis on 100 misclassified dev set examples from their image classifier. They discover that "Dog" mistakes account for 8% of the errors, while "Great Cat" and "Blurry" image errors make up the vast majority of the remaining errors.
Question: Based on Andrew Ng's guidelines for error analysis, what should the team diagnose or decide regarding their next steps to improve the classifier, and why?
Sample answer: The team should decide to focus their efforts on addressing the "Great Cat" or "Blurry" image errors rather than the "Dog" mistakes. Since Dog errors only account for 8% of the misclassifications, a project to fix them would eliminate at most 8% of the total errors. Working on the other categories provides a much larger opportunity to eliminate more errors.
Key points:
- Decide to focus on Great Cat or Blurry image errors.
- Working on Dog errors can eliminate 8% of the errors at most.
- Prioritizing higher-frequency categories helps eliminate more errors overall.
Rubric: The response must diagnose that the team should prioritize "Great Cat" or "Blurry" errors over "Dog" errors, justifying the decision by stating that fixing "Dog" errors has a strict upper limit of an 8% improvement.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
After error analysis on 100 misclassified dev set examples, Dog errors account for 8%. Which category should you likely prioritize?
Working on Dog classification mistakes can eliminate at most 8% of total errors, based on error analysis of 100 misclassified dev set examples.
Error analysis on 100 misclassified dev set examples shows that Dog mistakes can eliminate _____ of the errors at most.
Match each error category from Ng's example to the correct description of its impact on total errors.
Order the steps Ng recommends for using error category frequency to decide which error type to focus on.
In Ng's error analysis framework, what does an error category's percentage of misclassified examples represent?
According to Ng, you should always prioritize the error category that is easiest to fix, regardless of how often it appears in error analysis.
The percentage of misclassified dev set examples in a given error category represents the _____ percentage of total errors that could be eliminated by fixing it.
Match each term from Ng's error analysis framework to its correct definition.
Order the reasoning steps that explain why Ng concludes Dog errors should NOT be prioritized over Great Cat or Blurry image errors.
Explain How Error Category Frequency Guides Project Prioritization
Prioritizing Error Categories in a Vision Classifier
Maximum Impact of Fixing a Specific Error Category