Learn Before
Choosing the Size of a Monitoring Dev Set
Question: Answer in one to three sentences: What determines how many examples a training-development set should contain?
Sample answer: It should be large enough to give a dependable signal about whether the learning system is improving from one run to the next. In practice, it is often kept smaller than the full training set because its job is measurement, not model fitting.
Key points:
- Enough examples for a dependable evaluation
- Enough to compare progress across training runs
- Typically smaller than the training corpus
Rubric: The answer should connect the set size to reliable evaluation of training progress and may note that it is generally smaller than the training set.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Using Training, Training-Dev, Dev, and Test Sets for Different Questions
Sizing a Training Development Set
What is the key difference between a training dev set and the training set?
Should the development set reflect the training data distribution?
A validation set should not _____ on the data used to fit the model.
Match each split component to its purpose in model development.
Order the steps for setting up and using a training-dev split.
Why keep a development subset separate when it comes from the same data source?
Find the mistake in a team’s proposed holdout evaluation plan.
Choosing the Size of a Monitoring Dev Set
Which split best creates a training dev set?
Does a training-dev set need to be the same size as the training set?