Estimating Bias and Variance in a Crop Disease Classifier
Case context: You are building a crop disease classifier. The model has a 3% error rate on the training set and a 10% error rate on the development set. An experienced agronomist has about a 1% error rate.
Question: Using these error numbers, identify the main issue with the model. What machine learning terms describe this situation, and what do the 2% and 7% figures represent?
Sample answer: The main issue is high variance, also described as overfitting. The 2% gap between the training error (3%) and the human-level error (1%) is the estimated bias. The 7% gap between the development error (10%) and the training error (3%) is the estimated variance. Because the variance is much larger than the bias gap, the model is not generalizing well to new data.
Key points:
- Diagnose the problem as overfitting or high variance.
- Identify the 2% training-to-human gap as the estimated bias.
- Identify the 7% development-to-training gap as the estimated variance.
- Explain that the larger gap on the development set shows poor generalization.
Rubric: A correct response must diagnose overfitting or high variance. It should correctly label the 2% gap as estimated bias and the 7% gap as estimated variance.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Estimate the Variance from Training and Dev Errors
Training Success Does Not Guarantee Generalization
When a model fits the training set very closely but performs poorly on new data, that is called _____.
Match each concept to the description that fits a binary classifier’s error analysis.
Order the steps used to diagnose high variance in the traffic-sign classifier example.
Which training and development error pattern is most consistent with overfitting?
A model with high variance usually shows high training error and low development-set error.
Variance is computed from _____ minus training error.
Match each term to its role in diagnosing high variance in a document classifier.
Order the reasoning steps to decide whether a model is overfitting from its error numbers.
Interpreting a Classifier with Low Bias and High Variance
Term for Poor Generalization Despite Strong Training Performance
Related
Estimate the Variance from Training and Dev Errors
Training Success Does Not Guarantee Generalization
When a model fits the training set very closely but performs poorly on new data, that is called _____.
Match each concept to the description that fits a binary classifier’s error analysis.
Order the steps used to diagnose high variance in the traffic-sign classifier example.
Which training and development error pattern is most consistent with overfitting?
A model with high variance usually shows high training error and low development-set error.
Variance is computed from _____ minus training error.
Match each term to its role in diagnosing high variance in a document classifier.
Order the reasoning steps to decide whether a model is overfitting from its error numbers.
Interpreting a Classifier with Low Bias and High Variance
Term for Poor Generalization Despite Strong Training Performance