Rationale for Reviewing Correctly Classified Examples
Question: When improving dev set label quality, what is the primary reason for double-checking the labels of examples that your system has correctly classified?
Sample answer: The primary reason is that it is possible for both the original label and the learning algorithm's prediction to be simultaneously incorrect on the same example, making it appear as a correctly classified example.
Key points:
- Original labels can be wrong.
- Learning algorithms can be wrong.
- Both can be wrong on the exact same example.
Rubric: The response should explicitly mention the possibility that both the original label and the learning algorithm were wrong on the same example.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Bias from Fixing Labels of Only Misclassified Dev Examples
When improving dev set label quality, which examples should you double-check?
It is possible for both the original label and the learning algorithm to be wrong on the same dev example.
When improving label quality, double-check labels of both _____ and correctly classified dev examples.
Match each label-quality review scenario to its correct description.
Order the steps for conducting a thorough dev set label quality review per Machine Learning Yearning.
Why might a correctly classified dev example still contain a labeling error?
Reviewing only misclassified dev examples is sufficient for a complete label quality improvement process.
It is possible that both the original _____ and the learning algorithm were wrong on the same dev example.
Match each dev example category to its significance in the label quality review process.
Order the reasoning steps that justify reviewing correctly classified examples for label errors.
Discuss why it is insufficient to only review misclassified examples when improving dev set label quality.
Diagnosing Hidden Errors in a Cat Classifier Dev Set
Rationale for Reviewing Correctly Classified Examples