Learn Before
Which dataset should normally be used to compare model versions and decide whether to revert to an earlier checkpoint during development?
Reserve the final test set for one-time evaluation after model selection is finished.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
What is the main danger of using one holdout set over and over to decide releases, rollbacks, or model changes?
Which dataset should normally be used to compare model versions and decide whether to revert to an earlier checkpoint during development?
What happens if you repeatedly use the test set to choose among model variants?
When is it acceptable to check the test set during model development?
Using the test set to choose whether to keep a model rollback affects the test set's objectivity.
If you keep choosing models by looking at the test set, you begin to _____ the test set.
Classify Each Use of the Test Set as Safe or Unsafe
How Test-Set Feedback Can Make Evaluation Unreliable
Why should the test set stay out of model-tuning decisions?
Using the test set once a week to monitor overall progress is acceptable.
Once the test set has been used to compare models, it can no longer provide a fully _____ estimate of real-world performance.
Match each concept to its meaning when protecting a test set from being used during development.
Order the steps for using a test set without contaminating it during model development.
Why a weekly benchmark should not decide model rollbacks
Spotting the mistake in a model selection decision
What problem can occur if test-set scores are used to decide when to roll back a model version?