Learn Before
Why can training error go up even as dev error goes down?
Question: In a short explanation, describe why adding more training examples can make training performance look worse while improving development performance.
Sample answer: With a very small training set, a model may be able to memorize the examples almost perfectly, so training error can be very low. As the training set grows, the model must fit more cases, including unusual, noisy, or ambiguous ones, so training error often increases. Even so, the larger and more varied training set usually helps the model learn patterns that generalize better, so dev error tends to fall.
Key points:
- Small training sets are easier to memorize.
- Training error often rises as the training set gets larger.
- Noisy, ambiguous, or hard examples make perfect fitting more difficult.
- Dev error often decreases because the model generalizes better.
Rubric: A strong response explains both trends and links higher training error to reduced memorization and the greater difficulty of fitting a larger, more varied dataset, while also noting that dev performance typically improves.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What usually happens to training error as the training set becomes larger?
A very small training set can make a model seem impressively accurate on training data.
Training Error and More Data
Match each training-data situation with its typical effect on error.
Order the reasoning that explains why training error can increase.
Why can training error go up even as dev error goes down?
Explain the error trend after expanding a dataset
Why might a model get 0% training error on only two samples?
Which learning-curve trend is most typical when the training set gets larger?
Training error must always stay the same or decrease as the training set gets larger.