Learn Before
Growing Relative Impact of Mislabeled Dev Examples
As a system improves, mislabeled dev/test examples can become more important because their fraction grows relative to the total set of errors. When mislabeled dev examples add significant error to accuracy estimates, improving dev-set label quality becomes worthwhile.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Learn After
Why do mislabeled dev set examples become more impactful as a classifier improves?
It is acceptable to initially tolerate mislabeled dev/test examples and reconsider that decision as the system improves.
When mislabeled dev examples account for _____ of all errors, improving dev-set label quality becomes worthwhile.
Match each scenario to its correct implication regarding mislabeled dev set examples.
Order the reasoning steps for deciding whether to invest in fixing mislabeled dev set labels.
A classifier has ~2% dev error; 30% of those errors stem from mislabeled dev images. What should you do?
The difference between a classifier error of 1.4% and 2% is a minor detail with little practical significance.
As a classifier improves, the fraction of errors due to mislabeled dev examples _____ relative to total errors.
Match each concept to its role in the growing relative impact of mislabeled dev examples.
Order the stages of how mislabeled dev examples grow in importance across a classifier's development lifecycle.