Next Step After a Model Still Fits the Training Set Poorly
Case context: A team checks its classifier and finds that it performs badly even on the examples it was trained on. Earlier attempts to reduce simple underfitting have already been tried.
Question: State the diagnosis and describe the next analysis the team should do, including the style of procedure to follow.
Sample answer: The model is showing high bias, since it cannot match the training examples well. As a follow-up, the team should perform an error review on the training data itself. A useful way to do that is to use a process like the one used when examining mistakes on a dev set by eye.
Key points:
- Identify the problem as high bias.
- Link that diagnosis to poor fit on the training set.
- Recommend reviewing training examples for errors.
- Use a procedure modeled on manual dev-set error review.
Rubric: The response should name high bias, advise error analysis on the training set, present it as an additional step after earlier remedies, and mention a protocol similar to manual inspection of dev-set mistakes.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Sample the Worst Training Cases to Find Error Patterns
When is it useful to analyze errors on the training set?
A model with high bias is usually underfitting.
When bias is high, begin error analysis on the _____ set.
Match each diagnostic idea with its meaning in training analysis.
Order the steps for deciding whether to inspect training-set mistakes.
Why Train-Set Error Analysis Matters Under Underfitting
Next Step After a Model Still Fits the Training Set Poorly
Identifying High Bias Through Training Error Review
How does training-set error analysis fit into a high-bias debugging plan?
Training-set mistake analysis can use the same kind of procedure as dev-set mistake analysis.