Case Study

Choosing between two fixes after review of failure patterns.

Case context: A speech-to-text startup reviews recent mistakes in its transcription system. The audit shows that 45% of the recorded errors come from speakers with strong regional accents, while 12% come from short bursts of background noise. Improving accent handling would require collecting a new training set and redesigning parts of the network. Reducing noise-related mistakes could likely be done with preprocessing and a small set of augmentation steps.

Question: Using the idea of error analysis, what should the team consider when deciding which problem to tackle first? Why is the category with the largest share of errors not always the best immediate target?

Sample answer: The team should not choose the accent-related errors only because they make up the largest share of the failures. Error analysis helps identify important problem areas, but it does not provide a fixed rule for ranking them. The team should compare the likely benefit of each fix with the amount of time, data, and engineering effort it will take. In this case, the noise issue may be the better first project because it appears much easier to improve, so the team could gain useful progress more quickly even though it accounts for fewer errors.

Key points:

  • A larger share of mistakes does not by itself determine priority.
  • The likely gain from each improvement should be weighed against the effort required.
  • A smaller problem may be worth addressing first if it can be solved quickly and cheaply.

Rubric: The answer should explain that error analysis is a guide, not a strict ranking rule. It must state that the team should judge both expected improvement and implementation effort, and it should conclude that the easier noise-related fix may deserve attention before the harder accent-related overhaul even though it represents a smaller fraction of errors.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI