Diagnosing Variance in a Warehouse Package Sorter
Case context: You are evaluating a package-sorting system for a fulfillment center. Human performance on this task is nearly perfect, so the practical best error rate is about 0%. Your model achieves 2% error on the training set, 9% error on the training-dev set, and 9% error on the dev set.
Question: Based on these error metrics, what is the most likely diagnosis for the model's performance, and which comparison supports that diagnosis?
Sample answer: The most likely diagnosis is high variance. The key evidence is the jump from 2% training error to 9% training-dev error. Because the training error is already close to the ideal rate near 0%, the model does not appear to have a major bias problem. The much larger error on training-dev shows that it fits the training data better than it generalizes to new examples from the same distribution.
Key points:
- The correct diagnosis is high variance.
- The main evidence is the gap between 2% training error and 9% training-dev error.
- The low training error suggests bias is not the primary issue.
Rubric: The response must explicitly name high variance and justify it by comparing the 2% training error with the 9% training-dev error.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What main issue is suggested by 2% training error and 6% training-dev error when the best possible error is near 0%?
True or False: If training error is 2%, training-dev error is 9%, and the best achievable error is near 0%, the main issue is high bias.
A large gap between training error and training-dev error, with near-zero optimal error, indicates high _____.
Match the Error Metrics in a Variance Check
Order the steps for identifying the main error source when given training, training-dev, and dev error values.
Which gap points most directly to high variance when training error is 2%, training-dev error is 6%, and dev error is 6%?
True or False: In a diagnostic example of high variance, the model has 2% training error, 6% training-dev error, and 6% dev error.
The avoidable _____ is about 1% here because the training error (2%) is close to the best achievable error (~1%).
Match each diagnostic gap to the model issue it reveals in a retail demand forecast analysis.
Order the reasoning steps that support the conclusion that this model has high variance.
Interpreting Training and Training-Dev Errors for Variance
Diagnosing Variance in a Warehouse Package Sorter
How to Spot High Variance from a Gap in Errors