What problem is ruled out?
Question: A model has 8% error on the training set, 9% error on a training-development set, and 16% error on the dev set. This pattern points to high bias and a mismatch between the training source and the dev data. What issue is not suggested by these numbers?
Sample answer: It is not showing high variance on data from the same training distribution.
Key points:
- The training and training-development errors are close, so variance is not the main concern.
- The larger dev error points to a data mismatch rather than a variance problem.
Rubric: The answer must state that the model does not have high variance, especially on the training-data distribution.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What does a 1% gap between training error (14%) and training-dev error (15%) suggest?
True or False: An algorithm with 8% training error, 9% training-dev error, and 18% dev error is showing high variance on the training-set distribution.
High avoidable bias with a distribution shift
Match each error gap in the 8%/9%/17% scenario to the machine learning problem it diagnoses.
Order the diagnostic steps for identifying high bias and data mismatch without high variance.
Diagnosing Bias, Variance, and Distribution Shift from Error Rates
True or False: If training error is 8%, training-dev error is 9%, and dev error is 15%, the mismatch between training-dev and dev explains more of the drop than variance does.
Estimating Variance with a Held-Out Same-Source Set
Match each diagnosis to the evidence in the 12%/13%/19% scenario.
Order the reported error rates from lowest to highest in a case where the model fits the training set well but struggles on a shifted dev set.
Interpreting training, in-domain, and deployment errors
Diagnosing Bias and Distribution Shift
What problem is ruled out?