Essay

Why do different development and target data make model improvement harder to judge?

Question: Write a concise analytical response explaining how a gap between the development data and the final evaluation data affects interpretation of results and project priorities.

Sample answer: If the development set and the final evaluation set come from different populations, a gain on the development set may not mean the model will improve on the final data. That makes the development score a weaker signal for judging what actually helps. As a result, it becomes harder to tell which changes matter most and harder to decide what to work on next.

Key points:

  • The development and final evaluation data come from different distributions.
  • Improvement on the development set may not carry over to the final data.
  • This weakens confidence in the meaning of development-set results.
  • Lower confidence makes it harder to identify effective changes.
  • Unclear evidence makes prioritization more difficult.

Rubric: A strong response explains the mismatch, the uncertainty it creates about transfer to the final evaluation set, and the resulting difficulty in choosing priorities.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI