Why Recheck Apparently Correct Labels
Question: When cleaning labels in a validation set, why is it worth inspecting examples your classifier already marked correctly?
Sample answer: A case can look correct even when the recorded label and the model's prediction are both wrong in the same way. In that situation, the example is not actually evidence of a good label.
Key points:
- A stored label may be wrong.
- The model's prediction may also be wrong.
- If both are wrong in the same direction, the example appears correct.
Rubric: The response should state that a seemingly correct example may hide the fact that both the label and the model prediction are incorrect.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Bias from Correcting Only the Mistakes
Which examples should be reviewed to improve dev-set label quality?
A single development example can have both an incorrect target label and an incorrect model prediction.
Review Both Error Cases and Correct Cases
Match each label-audit situation to the correct description.
Order the steps for a dev-set label audit.
Why can a dev example appear to be labeled correctly even when the label is wrong?
To assess label quality, it is enough to inspect only the examples your model got wrong.
Reviewing Labels on Development Examples
Match each label-review category to its role in checking data quality.
Why Correct Predictions Still Need Label Review
Why checking only mistakes can miss label problems
Why Reviewing Only the Flagged Errors Can Miss Label Problems
Why Recheck Apparently Correct Labels