Case context: A company built a detector for identifying damaged boxes. Because it did not have access to images from the actual loading dock, the team trained the system on 58,000 catalog-style product photos collected from suppliers. After deployment, the detector performed much worse on live camera feeds from the dock than it had on the held-out test set.

Question: What is the most likely reason for the drop in performance, and what does it imply about the original choice to train on catalog photos?

Sample answer: The most likely reason is a mismatch between the training data and the real deployment data. Catalog photos are not a good proxy for loading-dock camera images, so the model can look strong on the test set but fail to generalize in the field. The original choice was still understandable if no better data were available, but the result shows that the team should work toward training and validation data that more closely match the real use case.

Key points:

• Catalog photos can differ greatly from live camera images
• A data mismatch can cause a generalization gap after deployment
• The original choice may have been reasonable if no representative data existed
• The outcome indicates a need for more target-like data

Rubric: Full credit: response identifies data mismatch as the reason for the gap and notes that the initial choice may have been reasonable given limited options. Partial credit: response names the mismatch but does not connect it to the deployment setting. No credit: response gives an unrelated explanation not supported by the case.