Learn Before
Choosing the most important error type to fix in a document classifier
Case context: You are building a classifier for scanned office documents. You review 120 development examples that your system misclassified. In the manual analysis, 48% of these errors are invoices, while 7% are handwritten notes.
Question: Based on the manual review of these 120 examples, which error type should you fix first, and how does the analysis support that choice?
Sample answer: You should prioritize invoice misclassifications. They make up 48% of the reviewed mistakes, compared with only 7% for handwritten notes. Fixing the most common error category is likely to reduce the overall development error the most.
Key points:
- Identify invoices as the highest-priority error category
- Use the 48% versus 7% counts from the review
- Explain that the largest category offers the biggest opportunity to lower total error
Rubric: The answer must identify invoices as the priority category and justify this choice by noting that they account for a much larger share (48% of the 120 misclassified examples) than handwritten notes (7%).
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why Perform Manual Error Analysis?
Should error analysis be limited to examples the model got right?
Manual error analysis is performed on about 100 incorrectly predicted _____ set examples.
Match each part of manual error review with its description.
Arrange the manual error-review process in order.
Using a Small Error Review to Set Priorities
Why inspect about 120 validation mistakes?
What should be sampled for error analysis?
True or False: Error analysis helps rank fixes by impact.
Related
Why Perform Manual Error Analysis?
Should error analysis be limited to examples the model got right?
Manual error analysis is performed on about 100 incorrectly predicted _____ set examples.
Match each part of manual error review with its description.
Arrange the manual error-review process in order.
Using a Small Error Review to Set Priorities
Why inspect about 120 validation mistakes?
What should be sampled for error analysis?
True or False: Error analysis helps rank fixes by impact.