Order the reasoning chain when the development set and test set come from different populations.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What remains uncertain when the validation and test sets come from different distributions?
A change in development and test distributions can make it harder to choose which model issue to fix first.
Different dev and test distributions add extra uncertainty to model evaluation
Match each distribution issue to its downstream consequence.
Order the reasoning chain when the development set and test set come from different populations.
Why do different development and target data make model improvement harder to judge?
Explain why a stronger dev result may not justify the next engineering priority.
Why can a validation-set gain be less trustworthy when the validation and test sets come from different distributions?
Why does a dev/test distribution mismatch make it harder to choose fixes?
A higher score on a mismatched development set guarantees better test performance.