Case context: A team says its model gets 72% accuracy on the training examples it was fit on and 89% accuracy on the dev/test examples it has never seen.

Question: Using only the principle described in the lesson, how should the team interpret this report, and what should it work on first?

Sample answer: The report should be treated as inconsistent with the principle. When the same evaluation setup is used, performance on unseen dev/test examples is usually worse than performance on the training set, not better. The team should first improve training-set performance before expecting good dev/test results.

Key points:

• The numbers describe an unlikely relationship between training and dev/test performance.
• Training examples are the ones the model has already seen.
• Dev/test examples are examples the model has not seen before.
• Dev/test performance is typically lower than training performance.
• The first priority is to improve training-set performance.

Rubric: Full credit requires noting that the report conflicts with the principle, distinguishing seen training data from unseen dev/test data, and stating that training-set performance should be improved first.