Google

When training error is already near the best possible level, there is little opportunity to reduce bias. If dev error remains much worse than training error, the main room for improvement is reducing variance.

High Variance Clue from Learning Curves

What learning-curve pattern usually indicates high variance?

When a model's training error is already near the best attainable error, there is usually still a large amount of bias reduction left.

Adding _____ to Improve Generalization

Match each learning-curve idea to its meaning.

Diagnosing a High-Variance Learning Curve

Question:
A classifier has a training error that is almost the best achievable on the training set, but its development-set error is much larger. Explain what this suggests about bias and variance, and where improvement is most likely to come from.

Sample answer:
Because the model is already near the best possible performance on the training data, it has little bias left to remove. The much larger development error means the model performs poorly on new examples, which is a sign of high variance. So the main opportunity is not to reduce bias, but to reduce variance. A practical way to do that is to use more training data, which often narrows the gap between training and development performance.

Key points:
- Near-best training error implies low bias.
- A much worse development error implies high variance.
- There is little remaining room to improve bias.
- There is substantial room to improve variance.
- More training data may reduce the generalization gap.

Rubric:
A strong response identifies low bias and high variance from the error pattern and explains that the likely gain comes from reducing variance rather than from further bias reduction.

Reading Bias and Variance from Error Rates

Case context:
A spam filter is being evaluated on a problem where the practical error floor is estimated to be about 9%. The model’s error on the training set is 9.3%, while its error on the validation set is 22%.

Question:
What does this pattern suggest about the model’s bias and variance, and which source of error has the most room for improvement?

Sample answer:
The training-set error is very close to the estimated error floor, so the model does not appear to have much bias left to remove. The much larger validation error shows that performance drops sharply on new data, which is a sign of high variance. Therefore, the main opportunity for improvement is variance reduction, for example by collecting more training examples or using stronger regularization.

Key points:
- Training performance near the error floor suggests low bias.
- A large gap between training and validation performance suggests high variance.
- Bias has little room for further improvement.
- Variance has substantial room for improvement.

Rubric:
The learner must identify low bias with little room to improve it, and high variance with substantial room to improve it.

Interpreting a Wide Train-Validation Gap

Question:
A classifier performs very well on its training data but much worse on new examples. Why is reducing overfitting likely to help more than trying to reduce bias?

Sample answer:
Because the model already fits the training set closely, the bias is probably small. The large gap between training performance and development performance shows that generalization is the main weakness, so there is more opportunity to reduce variance than to reduce bias.

Key points:
- Very low training error suggests limited bias.
- A large train-dev gap indicates high variance.
- The most promising gains come from reducing overfitting.

Rubric:
The answer must explain that near-optimal training performance leaves little bias to remove, while a large train-dev gap means variance reduction offers more improvement.

Learn Before

Related