Learn Before
Why Focus on the Incorrect Dev Predictions?
Question: In one to three sentences, explain why it is usually more efficient to examine only the wrong predictions in a development set instead of checking every prediction.
Sample answer: If a model is correct on most examples, the wrong predictions are a much smaller set to inspect. For example, if a system gets 97% accuracy on 2,500 items, only 75 items are incorrect, so reviewing those 75 takes far less time than reviewing the 2,425 correct ones.
Key points:
- A small number of errors is much easier to review than the full set.
- Less reviewing time means the analysis is more practical.
Rubric: Full credit requires noting the small error count compared with the much larger number of correct cases and explaining that this reduces review effort.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What is the main reason error-checking can become biased when revising labels in a dev set?
Reviewing the Mistakes
Counting Misclassifications in a Development Set
Match Each Validation-Set Quantity to Its Meaning
Why fixing only the mistakes can distort a development set
Explain why label checks often get focused on the examples a model gets wrong.
Explain the bias created when a team inspects only the validation errors.
Why Focus on the Incorrect Dev Predictions?
What problem can arise when you review only the validation examples the system gets wrong?
True or False: The 1,120 examples that were classified correctly must all have perfectly accurate labels because they were not selected for review.