A large gap with dev performance much better than test performance can indicate overfitting to the dev set.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Keep the Test Set Out of Routine Model Decisions
What makes a model gradually adapt itself to the dev set during development?
A large gap with dev performance much better than test performance can indicate overfitting to the dev set.
If repeated evaluation has made the old development set misleading, get a _____ development set.
Match each dev-set overfitting concept to its description.
Put the dev-set overfitting process in the correct order.
After several rounds of tuning, your validation score is much better than your test score. What should you do next?
It is a good practice to keep checking the test set after every major training change so you can choose the best model.
Repeatedly choosing models based on dev-set results can cause the system to gradually _____ to the dev set.
Match each development-set practice or signal with what it means in model selection.
Order the steps for investigating a validation-versus-test performance gap.
How Repeated Tuning Can Distort the Development Set
Evaluating a Model That Fits the Development Set Too Closely
Sign That the Development Set Has Been Overused