Question: What is the main purpose of error analysis when you suspect a mismatch between your training data and dev data? Why must you examine those two sets specifically?

Sample answer: The main purpose of error analysis in this situation is to identify how the training examples and the dev examples differ in ways that affect model behavior. Comparing those two datasets is important because the mismatch comes from those differences, not from the model alone. By studying them side by side, you can see which kinds of examples are underrepresented, overrepresented, or otherwise unlike what the model will face in evaluation or deployment.

Key points:

• Looks for differences between training data and dev data.
• Uses those differences to understand the source of the mismatch.
• Helps reveal which kinds of examples are causing the gap in performance.

Rubric: A strong response should say that error analysis is meant to uncover the important differences between the training set and the dev set, and that comparing those sets directly is necessary because those differences create the mismatch.