Diagnosing primary errors given 10% training, 11% training-dev, and 12% dev rates
Case context: You are evaluating a new image recognition model. Your logs show the following performance: a 10% error rate on the training set, an 11% error rate on the training-dev set, and a 12% error rate on the dev set.
Question: Based on these specific error metrics, what should you diagnose as the primary problem with this algorithm, and what does this imply about its current performance on the training data?
Sample answer: I would diagnose the model as having high avoidable bias. This implies that the algorithm is currently doing poorly on the training set, meaning it is underfitting or failing to capture the underlying patterns in the training data effectively.
Key points:
- Diagnose high avoidable bias.
- Conclude that performance on the training set is poor.
- Recognize that the model fails to learn the training data adequately.
Rubric: The response must correctly diagnose the issue as 'high avoidable bias' and explicitly state that this means the algorithm is performing poorly on the training set.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What does 10% training, 11% training-dev, and 12% dev error primarily indicate about a model?
An algorithm with 10% training error, 11% training-dev error, and 12% dev error is doing poorly on the training set.
An algorithm with 10% training, 11% training-dev, and 12% dev error has high _____ bias on the training set.
Match each error metric in ML Yearning's high avoidable bias scenario to its correct value or diagnosis.
Order the diagnostic steps for identifying high avoidable bias in ML Yearning's 10%/11%/12% error scenario.
In the 10%/11%/12% scenario, what does the 1% gap between training and training-dev errors tell us?
High variance (overfitting) is the primary error problem when training error is 10%, training-dev error is 11%, and dev error is 12%.
With 10% training, 11% training-dev, and 12% dev error, the algorithm is doing poorly on the _____ set.
Match each error problem type to the signal that most directly reveals it in the ML Yearning framework.
Order the reasoning steps for concluding that avoidable bias is the main problem when all three errors are clustered high.
Analyzing the root cause when training error is 10% and dev error is 12%
Diagnosing primary errors given 10% training, 11% training-dev, and 12% dev rates
Identifying the specific type of bias in the 10%/11%/12% error scenario