High Bias and Data Mismatch with a Small Training Gap
With 12% training error, 13% training-dev error, and 25% dev error, the algorithm has high avoidable bias and data mismatch, but not high variance on the training-set distribution.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
High Bias and Data Mismatch with a Small Training Gap
How can a single learning algorithm show avoidable bias, variance, and data mismatch at the same time?
True or False: A learning system can have both high avoidable bias and a data mismatch problem even if its variance is not high.
An algorithm can have any _____ of high avoidable bias, high variance, and data mismatch.
How can underfitting, overfitting, and distribution shift appear in the same model?
High Variance and Domain Mismatch Can Occur Together Without High Bias
A model may contain any _____ of three common error sources.
Match each error source to the comparison that best exposes it.
Order the checks used to diagnose bias, variance, and distribution shift.
A model has about the same error as expert performance on the training set, its training-dev error is nearly identical to its training error, and its dev-set error is much worse. What issue is most likely present?
If a classifier has high avoidable bias, it must also have high variance and a data mismatch problem.
A Large Training-Dev Gap Indicates Data Mismatch
Connect each error pattern to the combination of issues it signals.
Plan the Next Fix When Three Error Sources Are Present
Why Multiple Error Sources Can Appear at the Same Time
Diagnosing Multiple Error Sources in a Voice Transcription Model
Can Different Error Problems Appear Separately?
Learn After
What does a 1% gap between training error (14%) and training-dev error (15%) suggest?
True or False: An algorithm with 8% training error, 9% training-dev error, and 18% dev error is showing high variance on the training-set distribution.
High avoidable bias with a distribution shift
Match each error gap in the 8%/9%/17% scenario to the machine learning problem it diagnoses.
Order the diagnostic steps for identifying high bias and data mismatch without high variance.
Diagnosing Bias, Variance, and Distribution Shift from Error Rates
True or False: If training error is 8%, training-dev error is 9%, and dev error is 15%, the mismatch between training-dev and dev explains more of the drop than variance does.
Estimating Variance with a Held-Out Same-Source Set
Match each diagnosis to the evidence in the 12%/13%/19% scenario.
Order the reported error rates from lowest to highest in a case where the model fits the training set well but struggles on a shifted dev set.
Interpreting training, in-domain, and deployment errors
Diagnosing Bias and Distribution Shift
What problem is ruled out?