Learn Before
Why does the informal idea of variance require a comparison between training and held-out results?
Question: In a concise analytical response, explain why training performance by itself or held-out performance by itself does not fully capture the informal meaning of variance.
Sample answer: Informal variance refers to how much worse a model performs on a validation or test set than it does on the training set. That means the concept is based on a comparison between two performance levels. If you look at only one result, you do not see the performance gap that the term is meant to describe.
Key points:
- Training performance provides the baseline.
- Validation or test performance is compared against that baseline.
- Variance is the amount of degradation from training to held-out data.
- One performance number alone does not show the gap.
Rubric: A strong response identifies both parts of the comparison, explains that variance is the difference between them, and states why either result alone is incomplete.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What does variance mean in this informal comparison?
A larger performance gap between training data and validation data is an informal sign of variance.
Variance compares test performance to the ____ performance.
Match each term to its role in the informal variance idea.
Order the steps for judging whether a model has high variance
Why does the informal idea of variance require a comparison between training and held-out results?
Interpret a gap between training and validation performance.
What does “worse” mean in an informal variance comparison?
What information is needed to judge variance informally?
An informal variance score is defined as training-set error minus test-set error.