Essay

Why can training error go up even as dev error goes down?

Question: In a short explanation, describe why adding more training examples can make training performance look worse while improving development performance.

Sample answer: With a very small training set, a model may be able to memorize the examples almost perfectly, so training error can be very low. As the training set grows, the model must fit more cases, including unusual, noisy, or ambiguous ones, so training error often increases. Even so, the larger and more varied training set usually helps the model learn patterns that generalize better, so dev error tends to fall.

Key points:

  • Small training sets are easier to memorize.
  • Training error often rises as the training set gets larger.
  • Noisy, ambiguous, or hard examples make perfect fitting more difficult.
  • Dev error often decreases because the model generalizes better.

Rubric: A strong response explains both trends and links higher training error to reduced memorization and the greater difficulty of fitting a larger, more varied dataset, while also noting that dev performance typically improves.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI