Analyzing the root cause when training error is 10% and dev error is 12%
Question: In a scenario where an algorithm achieves a 10% training error, an 11% training-dev error, and a 12% dev error, analyze what this specific combination of error metrics indicates about the model's performance on the training set. Why does this pattern lead to this conclusion?
Sample answer: This combination of error metrics indicates that the algorithm has high avoidable bias. Because the training error is relatively high (10%) and the subsequent gaps to the training-dev error (1%) and dev error (1%) are small, the primary issue is that the algorithm is doing poorly on the training set itself, rather than struggling primarily with variance or overfitting.
Key points:
- Identifies high avoidable bias.
- States the algorithm is doing poorly on the training set.
- Notes the small gaps between error metrics indicate variance is a secondary concern.
Rubric: Answers should identify high avoidable bias and explain that the core problem is poor performance on the training set, noting that the gaps between the training, training-dev, and dev errors are small compared to the initial training error.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What does 10% training, 11% training-dev, and 12% dev error primarily indicate about a model?
An algorithm with 10% training error, 11% training-dev error, and 12% dev error is doing poorly on the training set.
An algorithm with 10% training, 11% training-dev, and 12% dev error has high _____ bias on the training set.
Match each error metric in ML Yearning's high avoidable bias scenario to its correct value or diagnosis.
Order the diagnostic steps for identifying high avoidable bias in ML Yearning's 10%/11%/12% error scenario.
In the 10%/11%/12% scenario, what does the 1% gap between training and training-dev errors tell us?
High variance (overfitting) is the primary error problem when training error is 10%, training-dev error is 11%, and dev error is 12%.
With 10% training, 11% training-dev, and 12% dev error, the algorithm is doing poorly on the _____ set.
Match each error problem type to the signal that most directly reveals it in the ML Yearning framework.
Order the reasoning steps for concluding that avoidable bias is the main problem when all three errors are clustered high.
Analyzing the root cause when training error is 10% and dev error is 12%
Diagnosing primary errors given 10% training, 11% training-dev, and 12% dev rates
Identifying the specific type of bias in the 10%/11%/12% error scenario