True or False: If a model scores lower on the test set than on the dev set, then the test set must be inherently more difficult.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Why a Mismatched Validation Set Can Waste Effort
Which explanation is NOT one of the standard reasons a model can score well on the dev set but poorly on the test set when the two sets come from different distributions?
True or False: If a development set and a test set come from different distributions, it is easy to pinpoint the reason a model scores worse on the test set.
If the development set and the test set come from different _____, a performance gap is hard to interpret.
Match each distribution-shift situation to the most likely explanation.
Order the three explanations for why a model can look strong on development data but weak on test data when the data distributions are different.
When the test data are drawn from a tougher distribution than the development data, what is the most reasonable conclusion?
True or False: If a model scores lower on the test set than on the dev set, then the test set must be inherently more difficult.
A model may do well on the _____ set and still perform poorly on the test set if the two sets come from different distributions.
Match each dev/test diagnosis to the most appropriate next step.
Arrange the steps a practitioner follows when a spam filter succeeds on validation data but fails after deployment.
Why a good development score can still leave test results unexplained
Why a Model Looks Strong in Validation but Weak on Deployment Data
Three Reasons a Held-Out Test Set Can Look Worse