Case Study

Diagnosing a speech command classifier with high training and dev error

Case context: You are building a voice-command classifier that should reach about 4% error. Right now, it has 14% error on the training set and 15% error on the dev set. A teammate suggests collecting 8,000 more labeled recordings before making any other changes.

Question: Using the error rates and target, identify the main issue, assess the proposed fix, and state what should be addressed first to improve the system.

Sample answer: The main issue is avoidable bias, because the training error of 14% is far above the 4% target. Adding 8,000 more recordings is unlikely to solve the problem, since the training and dev errors are already very close, which means variance is not the main limitation. More data can help when the model overfits, but here the bigger need is to improve performance on the current training set. The first step should be to make the learning algorithm fit the training data better before focusing on additional data collection.

Key points:

  • Identify avoidable bias as the primary issue because 14% training error is much higher than the 4% goal.
  • Explain that collecting more data is not the best first move because it mainly helps with variance, not bias.
  • Recommend improving training-set performance first.

Rubric: Diagnose avoidable bias based on the 14% training error versus the 4% target error. Explain why collecting more data is ineffective as a first response because it mainly addresses variance rather than bias. Recommend improving the model’s performance on the training set first.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI