Learn Before
Interpret a gap between training and validation performance.
Case context: A team trains a spam filter that scores better on its training emails than on its validation emails. The team wants to understand what that gap means.
Question: What should the team compare, and what does the size of the drop indicate?
Sample answer: The team should compare the model's validation performance with its training performance. The amount by which validation performance is worse than training performance is informally called the model's variance.
Key points:
- Compare validation performance against training performance.
- Focus on how much worse the validation result is.
- Treat that drop as an informal measure of variance.
Rubric: The response should name both training and validation performance, keep the comparison direction correct, and identify the performance drop on validation as informal variance.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What does variance mean in this informal comparison?
A larger performance gap between training data and validation data is an informal sign of variance.
Variance compares test performance to the ____ performance.
Match each term to its role in the informal variance idea.
Order the steps for judging whether a model has high variance
Why does the informal idea of variance require a comparison between training and held-out results?
Interpret a gap between training and validation performance.
What does “worse” mean in an informal variance comparison?
What information is needed to judge variance informally?
An informal variance score is defined as training-set error minus test-set error.