Prompt: A classifier has 12% error on its training set, 13% error on a separate set drawn from the same source as the training data, and 23% error on the development set. Assume human-level performance is essentially perfect. What do these numbers say about the model's main problems? State which issues are present and which are not, and justify your answer from the error gaps.

Model answer: This pattern indicates avoidable bias and a data mismatch problem, but not substantial variance on the training-data distribution. The 12% training error is already high, so the model is underfitting relative to near-perfect human performance. The tiny 1% increase from training to the same-source holdout set shows the model generalizes about as well there as it does on the training data, so there is no strong sign of high variance on that distribution. The much larger jump from 13% to 23% on the development set suggests that the development data differs from the data the model was tuned on, which is evidence of data mismatch.

Key points:

• High avoidable bias is indicated by the 12% training error.
• No strong high-variance problem is indicated because the training and same-source holdout errors are close (12% vs. 13%).
• Data mismatch is indicated by the much larger gap between the same-source holdout error and the development error (13% vs. 23%).

Rubric: The response must identify avoidable bias, data mismatch, and the absence of major variance. It should support each claim using the specific differences among the three error rates.