What a learning curve suggests when both underfitting and overfitting are present
Question: Explain what the training and development errors typically look like when a model has both high bias and high variance, and state the broad direction for improving the model.
Sample answer: When a model has high bias, the training error stays high, so performance on the training set is still far from what we want. When it also has high variance, the development error is much worse than the training error, showing that the model does not generalize well. Seeing both patterns at once means the model needs changes that lower both underfitting and overfitting, not just one or the other.
Key points:
- Training error remains much worse than desired, which signals high bias
- Development error is much higher than training error, which signals high variance
- The model must be improved in ways that address both problems
Rubric: A strong response must describe the high training error, the large gap between development and training error, and the need to reduce both bias and variance.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Recognizing When Both Bias and Variance Are High
If a model has both high bias and high variance, its training error and development error are usually about the same.
Learning Curves That Point to _____
Reading Training and Validation Patterns
Diagnosing Bias and Variance in a Model
What a learning curve suggests when both underfitting and overfitting are present
Evaluating a Sales Forecast Model
What does a large validation gap indicate when training error is already high?
What should you do when the learning curve suggests both weak fit and high instability?
When a classifier has high bias, it usually achieves very low training error.