Learn Before
How a training-set increase should affect dev error
Case context: A team building a fraud-detection model first trained it on 400 labeled transactions and measured a 30% dev-set error. They then retrained the same model using 1,600 examples and later 6,400 examples, while keeping the dev set unchanged.
Question: According to the general learning-curve relationship in machine learning strategy, what should happen to the dev-set error as the team moves from 400 to 1,600 and then 6,400 training examples?
Sample answer: The dev-set error should trend downward as the training set grows. A larger training set should generally produce a model that performs better on the dev set than the version trained on only 400 examples.
Key points:
- Increasing the training-set size should reduce dev-set error.
- The model trained on more examples should be evaluated as having better dev performance.
Rubric: The student must correctly state that dev-set error is expected to decrease as the number of training examples increases from 400 to 6,400.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What trend should you expect in dev-set error as the training set gets larger?
True or False: Increasing the training set often reduces dev-set error when the model can benefit from more data.
More Training Data and Dev Error
Match each learning-curve term to its description.
Order the steps for creating a validation learning curve.
A team trains on 2,500 examples and sees 24% dev-set error. After adding 7,500 more training examples, what outcome should they expect?
True or False: Adding more training examples usually makes dev-set error worse.
What a Learning Curve Plots
Match each learning-curve pattern to its most likely interpretation.
Order the steps for deciding whether more training data is worth collecting based on a dev-set learning curve.
Why does validation error usually fall as the training set gets larger?
Effect of more training data on validation error
Related
What trend should you expect in dev-set error as the training set gets larger?
True or False: Increasing the training set often reduces dev-set error when the model can benefit from more data.
More Training Data and Dev Error
Match each learning-curve term to its description.
Order the steps for creating a validation learning curve.
A team trains on 2,500 examples and sees 24% dev-set error. After adding 7,500 more training examples, what outcome should they expect?
True or False: Adding more training examples usually makes dev-set error worse.
What a Learning Curve Plots
Match each learning-curve pattern to its most likely interpretation.
Order the steps for deciding whether more training data is worth collecting based on a dev-set learning curve.
Why does validation error usually fall as the training set gets larger?
Effect of more training data on validation error