A dev set should always be made as large as possible, even beyond what meaningful-change detection requires.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Small Dev Sets Cannot Detect Tiny Accuracy Differences
Common Dev Set Sizes for Detecting 0.1% Improvements
High-Value Applications May Need Larger Dev Sets
Statistical Significance Tests on Dev Set Changes
Which dev set best supports detecting a 0.1 percentage-point accuracy improvement?
A dev set should always be made as large as possible, even beyond what meaningful-change detection requires.
A dev set of _____ examples gives a good chance of detecting a 0.1% improvement.
Match each accuracy-detection goal with its dev set implication.
Order the reasoning process for sizing a dev set around a meaningful accuracy change.
Explain how the size of a meaningful accuracy improvement should influence dev set size.
Decide whether a mature recommendation system needs a larger dev set.
Why is a 100-example dev set inadequate for comparing 90.0% and 90.1% accuracy?
When is a dev set much larger than 10,000 examples most justified?
A team satisfied with detecting meaningful changes need not enlarge its dev set far beyond that requirement.