Learn Before
Diagnosing a Dev Error Gap
Case context: You are reviewing a learning curve for a model where the optimal error rate is known to be around 14%. Your algorithm's training error is currently at 14.5%, but its dev error is at 26%.
Question: Based on this learning curve, what should you diagnose about the model's bias and variance, and where is the room for improvement?
Sample answer: The model's training error of 14.5% is very close to the optimal error rate of 14%, indicating that the bias is small and there is not much room for improvement in terms of bias. However, the dev error of 26% is much higher than the training error, showing the model is not generalizing well. This indicates high variance, meaning there is ample room for improvement in errors due to variance, potentially by adding more training data.
Key points:
- Training error near optimal implies small bias.
- A large gap between dev error and training error implies large variance.
- There is little room to improve bias.
- There is ample room to improve variance.
Rubric: The learner must diagnose low bias with little room for bias improvement, and high variance with ample room for variance improvement.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Identifying High Variance from Learning Curves
Room for Bias Improvement
Adding _____ to Close the Error Gap
Matching Learning Curve Concepts
Analyzing a High Variance Learning Curve
Assessing Error Improvement Potential
Diagnosing a Dev Error Gap
Variance vs. Bias Improvement Potential
Effect of Adding Training Data
Generalization and High Variance