Question: A text classification model is trained on articles from one publication, tuned on a held-out set from the same source, and then tested on posts from a different platform. Explain how it could show three problems at once: weak fit to the training data, a large gap between training and tuning performance, and lower performance caused by the new platform’s data distribution. Why would improving one problem not necessarily fix the others?

Sample answer: These problems are measured by different comparisons, so they are separate rather than mutually exclusive. Weak fit to the training set shows that the model is not capturing the training pattern well. A large training-versus-tuning gap indicates instability or overfitting on the training split. Lower performance on the new platform reflects a distribution shift between the data used for development and the data used in deployment or evaluation. A model can have all three issues at once because each one comes from a different source. For example, a model may be too simple for the training set, overly sensitive to training examples, and still face a different data distribution at test time. Increasing model capacity may improve the first problem, but it will not automatically remove the distribution shift or the overfitting gap.

Key points:

• The three problems are distinct and measured using different performance comparisons.
• Any combination of the three can occur in the same system.
• Fixing one issue does not guarantee that the others will improve.

Rubric: The response must explain that weak training fit, the training-tuning gap, and distribution shift are separate sources of error measured by different comparisons. It must state that they can occur together in any combination and explain that correcting one source does not automatically solve the others.