Short Answer

Why choose dev and test sets from the same distribution instead of building a special adaptation method?

Question: A machine learning team is deciding between two priorities: (1) making the dev and test sets match the data they care about most, or (2) investing time in a domain adaptation method for one particular deployment setting. According to the general strategy taught in machine learning practice, why is the first option often the better immediate choice?

Sample answer: Using dev and test sets drawn from the same distribution the team wants to improve on usually leads to faster progress. It gives clear feedback on whether a change helps the main task. By contrast, domain adaptation is a harder, research-heavy problem, and many techniques work only in limited situations, so it is not usually the best first route for everyday product improvement.

Key points:

  • Same-distribution dev and test sets help the team move more efficiently.
  • Domain adaptation is a research-oriented problem with limited generality.

Rubric: The response must mention improved team efficiency and the specialized, research-focused nature of domain adaptation.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI