Case Study

Fix a training-dev distribution gap

Case context: You are building a defect detector for used battery packs. The model achieves strong results on examples collected in a controlled lab, where the batteries are clean and evenly lit. But when you evaluate it on the development set, most items come from a busy warehouse line with dust, shadows, and inconsistent camera angles, and the error rate is much higher.

Question: What problem is the system experiencing, and what focused action should you take to help the algorithm handle the hard development examples?

Sample answer: This is a data distribution mismatch: the examples used to train the model are not similar to the examples it struggles with during development. The best next step is to collect additional labeled battery-pack images from the warehouse setting and include them in the training set so the model sees more of the same kind of data.

Key points:

  • Identify the issue as a mismatch between the training set and the development set.
  • Recognize that the lab images and warehouse images come from different distributions.
  • Recommend adding more training examples that resemble the difficult development cases.
  • State that the added examples should come from the warehouse environment.

Rubric: Responses must correctly identify the issue as a data distribution mismatch and recommend the specific action of collecting more warehouse-style training images.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI