Case context: A team downloads website pictures of cats and non-cats, splits them into training and test sets, and obtains strong results. After adding the detector to a mobile app, it performs poorly on pictures uploaded by users. Those pictures tend to be lower resolution, blurrier, and poorly lit.

Question: What should the team diagnose as the central problem, and why did its earlier evaluation fail to reveal it?

Sample answer: The central problem is a mismatch between the website-image distribution used for training and testing and the mobile-phone-picture distribution encountered in the app. The earlier evaluation failed to reveal the problem because its test images came from the same website-image source rather than from the actual user-upload distribution.

Key points:

• Website images supplied the development data.
• Mobile-phone pictures supplied the deployed inputs.
• The two sources differed in visible characteristics.
• The original test set did not represent the actual distribution of interest.

Rubric: Full credit requires naming the distribution mismatch, distinguishing website images from mobile-phone uploads, and explaining why a website-based test set could not reliably indicate performance on the deployed distribution.