Learn Before
Error Analysis
Error analysis is the practice of studying a model’s mistakes to decide what should be improved next. There is no single mandatory procedure; teams often use familiar patterns, but they can also adapt the approach to fit the problem and try other ways of examining errors.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Single-Score Model Evaluation
An Early Dev/Test Setup Speeds Model Iteration
Revise the evaluation setup when it stops matching the goal
Error Analysis
Reviewing Both Missed and Correct Dev-Set Examples
After a development set and a test set have been chosen, what are they mainly used for?
True or False: A development set score can provide a quick check on whether a new idea is moving a model in the right direction.
Dev and test sets give a quick check on how well a team’s _____ is performing.
Match each dev/test-set action with its main effect.
Arrange the basic workflow for using development and test sets to improve a model.
Explain why a dev set, test set, and metric help a machine learning team work efficiently.
Diagnosing a project with no evaluation set
In one to three sentences, explain how a validation set can guide which model ideas deserve more work.
How does dev set performance help a team choose which ideas to keep improving?
What do teams usually try once dev and test sets are fixed?
What do teams typically do after they have set up dev and test sets?
True or False: With a fixed development set and a single evaluation metric, a team can quickly tell whether a new idea is helping by a little or by a lot.
A development set and a chosen evaluation metric help a team compare ideas quickly and see whether each change is making progress.
Learn After
One Example May Fit Several Error Tags
New Error Categories Can Appear During Review
Choose Error Categories You Can Act On
Error Review Improves Through Repeated Passes
Using Error Counts to Decide Where to Focus Next
Working on Several Error Buckets at Once
Error Analysis Is Not an Automatic Ranking Rule
A Category's Share of Errors Sets an Upper Bound on Improvement
Error Analysis Helps Estimate Whether a Proposed Change Is Worth the Effort
Why Quick Error Review Is Often Skipped
Incorrect Labels in a Validation Set
Splitting a large development set into a review subset and a tuning subset
Build a Simple Baseline First, Then Use Error Analysis to Prioritize Improvements
Using Training-Set Mistakes to Diagnose High Bias
Reviewing a Sample of Validation Errors
Separating Search Errors from Scoring Errors in Inference
Component-Wise Error Review
Error Analysis as a Data-Science Lens on Model Mistakes
Multiple Valid Approaches to Error Analysis
Tasks Humans Can Perform Give Stronger Error Analysis Benchmarks
When diagnosing a model, what should error analysis focus on first?
Error analysis on a machine learning system must follow one fixed procedure.
Name the practice of reviewing mistakes to understand why predictions failed.
Match each error-analysis idea to the description that best fits it.
Put the steps of a simple dev-set error review in the right order.
Why is it useful to inspect misclassified examples during error analysis, even for error types you cannot immediately repair?
Error analysis is usually repeated after each round of model changes.
Error analysis can help you judge which improvement paths look most _____.
Match each error-analysis activity with the benefit it can provide.
Order the steps for deciding which error types to target after a first pass of error review.
Why Error Review Helps Set the Right Next Priorities
Plan the next review step after repeated image-classifier mistakes.
What is error analysis used for in machine learning?