As training-set size grows, dev and test error should decrease, but training-set error usually increases. With very small training sets, an algorithm can often memorize the examples and get near-zero training error; with larger and more ambiguous or mislabeled training sets, perfectly fitting every example becomes harder.

Google

If one only looks at the dev-error curve, it can be hard to extrapolate and predict exactly where the curve will go with more data. Training error is an additional plot that can help estimate the impact of adding more data.

Training Error Plot for Estimating the Effect of More Data

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Training Error Usually Increases with Training Set Size

A learning algorithm usually performs better on the training set than on the dev set, so the dev-error curve usually lies above the training-error curve.

Learn Before

Related