Learn Before
Case study: Updating an error analysis spreadsheet when some labels may be wrong.
Case context: A machine learning developer is building a seal-versus-otter image classifier. During error analysis, they suspect that some validation examples have incorrect labels, which could distort the measured error rates. They want to decide whether the amount of mislabeled data is large enough to justify a broader cleanup.
Question: What should the developer add to the error analysis spreadsheet to examine this issue, and what columns should the spreadsheet contain?
Sample answer: The developer should add a 'Wrong label' category so they can measure how much of the validation set is mislabeled. The spreadsheet should have these six columns: File, Seal, Otter, Motion blur, Wrong label, and Notes.
Key points:
- Add a 'Wrong label' category to track the fraction of mislabeled examples.
- Use the six columns: File, Seal, Otter, Motion blur, Wrong label, and Notes.
Rubric: A correct response must say that a 'Wrong label' category should be included to track mislabeled examples. It must also list the six columns: File, Seal, Otter, Motion blur, Wrong label, and Notes.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
When should a review log include a separate bucket for likely data issues?
When to mark an example as mislabeled
When you suspect many development examples may have the wrong label, what kind of spreadsheet column should you add to flag them?
When should you add a label-quality column to your error log?
A mislabeled entry in an error log marks cases where the reference label is wrong.
Add a mislabeled category to the error-analysis sheet when the _____ of incorrect labels appears to be substantial.
Match each spreadsheet column name to the kind of error information it records.
Order the Steps for Adding a Wrong-Label Column to an Error Review Spreadsheet
Which column is not part of the digit review log?
A mislabeled column belongs in every error-analysis spreadsheet by default.
Audit Sheet Column for Incorrect Labels
Match each description to the correct column in the review log.
Order the reasoning steps that lead a practitioner to add a 'Label error' column rather than ignoring suspected bad labels.
When to Add a Mislabeled Category to Error Analysis
Case study: Updating an error analysis spreadsheet when some labels may be wrong.
When to Add a Mislabeled Column