Case Study

Evaluating a Sales Forecast Model

Situation: A team is checking a model that predicts weekly store demand. Their target error rate is 5%. On the training set, the model’s error has leveled off at 18%. On the development set, the error is 31%.

Question: What do these results suggest about the model’s bias and variance, and what should the team try to do next?

Sample answer: The model shows high bias because its training error is still far above the target level. It also shows high variance because the development error is much worse than the training error. The next step is to work on reducing both bias and variance.

Key points:

  • High bias is indicated by 18% training error compared with a 5% target.
  • High variance is indicated by 31% development error compared with 18% training error.
  • The recommended direction is to reduce both sources of error.

Rubric: The response must correctly identify high bias from the large training error, identify high variance from the training-versus-development gap, and recommend reducing both.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI