Case context: A spam filter is being evaluated on a problem where the practical error floor is estimated to be about 9%. The model’s error on the training set is 9.3%, while its error on the validation set is 22%.

Question: What does this pattern suggest about the model’s bias and variance, and which source of error has the most room for improvement?

Sample answer: The training-set error is very close to the estimated error floor, so the model does not appear to have much bias left to remove. The much larger validation error shows that performance drops sharply on new data, which is a sign of high variance. Therefore, the main opportunity for improvement is variance reduction, for example by collecting more training examples or using stronger regularization.

Key points:

• Training performance near the error floor suggests low bias.
• A large gap between training and validation performance suggests high variance.
• Bias has little room for further improvement.
• Variance has substantial room for improvement.

Rubric: The learner must identify low bias with little room to improve it, and high variance with substantial room to improve it.