Case Study

Diagnosing a Classifier with 15% Training Error and 30% Dev Error

Case context: You train a classifier and find that it achieves a 15% error rate on the training set, which you estimate as the bias. When evaluated on the dev set, the error rate increases to 30%, which represents an estimated variance of 15%.

Question: Based on Andrew Ng's concepts, diagnose the bias and variance of this classifier. Explain how it performs on each dataset and why the traditional labels of overfitting or underfitting are problematic here.

Sample answer: The classifier has both high bias and high variance. It performs poorly on the training set (15% bias) and performs even worse on the dev set (15% variance, 30% total error). In this scenario, applying either 'overfitting' or 'underfitting' terminology is problematic because both issues are happening simultaneously.

Key points:

  • Identify the classifier as having both high bias and high variance.
  • Explain poor training set performance represents high bias and even worse dev set performance represents high variance.
  • State that overfitting and underfitting are occurring simultaneously, making standard labels problematic.

Rubric: Evaluates whether the student correctly identifies the classifier as having both high bias and high variance, links these to poor training performance and even worse dev set performance, and states that overfitting and underfitting are occurring simultaneously.

0

1

Updated 2026-06-13

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Machine Learning Strategy

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Yearning @ DeepLearning.AI