Learn Before
Beginning error review without a checklist
Question: When you start reviewing mistakes in a model’s output, is it a problem if you do not already have a set of error labels in mind? Briefly describe the best first move for creating those labels.
Sample answer: No. It is fine to begin with no labels. The best first step is to examine a small number of incorrect examples; the patterns in those cases usually suggest the first categories worth tracking.
Key points:
- Starting without labels is acceptable.
- Inspect a few errors first.
- Use those examples to propose initial categories.
Rubric: 1 point for saying it is okay to begin without labels; 1 point for explaining that a small set of errors should be reviewed first to generate category ideas.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What best describes how error analysis should be carried out on a machine learning model?
True or False: You must decide every error label in advance before examining model mistakes.
Grouping mistakes can reveal patterns
Match each stage of an iterative mistake-review cycle to its description.
Arrange the stages of one error-analysis cycle in the correct order.
You find a new mistake pattern while reviewing model outputs for the first time. What is the best next step?
True or False: If error analysis uncovers a new failure type, earlier examples may need to be reviewed again under that new category.
After a second pass reveals new mistake patterns, inspect the same examples using the _____ categories.
Match each property of error review to the label that best describes it.
Arrange the steps that show why error analysis works best as a repeating process.
Why does error analysis often require repeated passes?
Adding a New Error Category During Review
Beginning error review without a checklist