Why must training performance be solid before you expect strong validation or test performance?
Question: Respond in one to three sentences by relating performance on data the model has already learned from to performance on new data.
Sample answer: A model is judged first on the data it was trained with, because that shows whether it has learned the task at all. If it does not do well on familiar training examples, there is little reason to expect it to do well on new validation or test examples, which are usually harder.
Key points:
- Training examples are the data used to fit the model.
- Results on validation or test data are typically lower than training results.
- Strong learning on the training set should come before expecting good generalization.
Rubric: The response should refer to training examples as the data already seen, note that validation or test results are usually worse, and explain that strong training performance should come first.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What usually has to happen before a model is expected to do well on validation and test data?
True or False: Performance on development and test data is usually lower than performance on the training set.
Training performance comes first
Interpreting Training and Held-Out Performance
Order the steps used to judge whether a better score on unseen data is believable.
Explain why training performance must come before strong validation or test performance.
Diagnose a claim of 72% training accuracy and 89% dev/test accuracy.
Why must training performance be solid before you expect strong validation or test performance?
Which reported result contradicts the training-first rule?
A model that fits the training set poorly can still be expected to do well on dev or test data.