Learn Before
Double-Checking Labels of Both Misclassified and Correctly Classified Dev Examples
When deciding to improve label quality, consider double-checking the labels of both examples that the system misclassified and examples it correctly classified. It is possible that both the original label and the learning algorithm were wrong on an example.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Single-Number Evaluation Metric
Initial Dev/Test Set and Metric Helps Quick Iteration
Changing Dev/Test Sets or the Metric When They No Longer Guide the Team
Error Analysis
Double-Checking Labels of Both Misclassified and Correctly Classified Dev Examples
What is the primary purpose of the dev and test sets once they are defined?
True or False: Measuring an idea's performance on the dev set lets a team quickly decide if it is heading in the right direction.
The dev and test sets allow a team to quickly see how well their _____ is doing.
Match each action related to dev/test sets with its described effect.
Order the steps of using dev/test sets to guide iteration on a machine learning idea.
Explain why having dev/test sets and a metric is important for a team's workflow.
A team tries several algorithm changes without a dev set. Diagnose the resulting problem.
In one to three sentences, describe how the dev set helps a team decide which ideas to keep refining.
In one to three sentences, describe how the dev set helps a team decide which ideas to keep refining.
What kind of things might a team try after defining a dev set and test set?
What kind of things might a team try after defining a dev set and test set?
True or False: Having a dev set and metric allows a team to quickly detect which ideas give small or large improvements.
True or False: Having a dev set and metric allows a team to quickly detect which ideas give small or large improvements.
Learn After
Bias from Fixing Labels of Only Misclassified Dev Examples
When improving dev set label quality, which examples should you double-check?
It is possible for both the original label and the learning algorithm to be wrong on the same dev example.
When improving label quality, double-check labels of both _____ and correctly classified dev examples.
Match each label-quality review scenario to its correct description.
Order the steps for conducting a thorough dev set label quality review per Machine Learning Yearning.
Why might a correctly classified dev example still contain a labeling error?
Reviewing only misclassified dev examples is sufficient for a complete label quality improvement process.
It is possible that both the original _____ and the learning algorithm were wrong on the same dev example.
Match each dev example category to its significance in the label quality review process.
Order the reasoning steps that justify reviewing correctly classified examples for label errors.
Discuss why it is insufficient to only review misclassified examples when improving dev set label quality.
Diagnosing Hidden Errors in a Cat Classifier Dev Set
Rationale for Reviewing Correctly Classified Examples