Question: Why does a mismatch between the dev set and the test set make it harder for a machine learning team to decide what to improve next?

Sample answer: Because it becomes unclear whether a change that helps the dev set will also help the test set. Without that link, the team cannot reliably tell which fixes are truly improving overall performance, so choosing the next task becomes much harder.

Key points:

• A mismatch weakens the connection between dev-set gains and test-set gains.
• It makes it harder to judge which changes are genuinely useful.

Rubric: The answer should explain that the mismatch creates uncertainty about whether dev-set improvements transfer to the test set, which makes prioritizing work more difficult.