Learn Before
Choosing the most important error type to fix in a document classifier
Case context: You are building a classifier for scanned office documents. You review 120 development examples that your system misclassified. In the manual analysis, 48% of these errors are invoices, while 7% are handwritten notes.
Question: Based on the manual review of these 120 examples, which error type should you fix first, and how does the analysis support that choice?
Sample answer: You should prioritize invoice misclassifications. They make up 48% of the reviewed mistakes, compared with only 7% for handwritten notes. Fixing the most common error category is likely to reduce the overall development error the most.
Key points:
- Identify invoices as the highest-priority error category
- Use the 48% versus 7% counts from the review
- Explain that the largest category offers the biggest opportunity to lower total error
Rubric: The answer must identify invoices as the priority category and justify this choice by noting that they account for a much larger share (48% of the 120 misclassified examples) than handwritten notes (7%).
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why Perform Manual Error Analysis?
Should error analysis be limited to examples the model got right?
Manual error analysis is performed on about 100 incorrectly predicted _____ set examples.
Match each part of manual error review with its description.
Arrange the manual error-review process in order.
Using a Small Error Review to Set Priorities
Choosing the most important error type to fix in a document classifier
Why inspect about 120 validation mistakes?
What should be sampled for error analysis?
True or False: Error analysis helps rank fixes by impact.