Case context: A team partners with a video game company to generate 100,000 realistic synthetic images of cars using the game's graphics models, which include about 20 distinct car designs. The team trains a computer vision model on these images and achieves very high accuracy on their training set, but the model performs poorly on a dev set containing photos of real cars on roads.

Question: What is the most likely cause of the poor dev set performance, and what does this reveal about the relationship between training data volume and data diversity?

Sample answer: The most likely cause is overfitting to the limited set of ~20 car designs present in the synthetic data. Even though the training set has 100,000 images and achieves high training accuracy, all those images share the same small set of underlying car shapes, so the model has effectively memorized features specific to those 20 designs rather than learning generalizable car-recognition features. The real-world dev set includes a much wider variety of car designs, so the model's narrow learned features fail to transfer. This reveals that having many training examples does not compensate for low underlying diversity; what matters is whether the training distribution adequately represents the distribution the model will be tested and deployed on.

Key points:

• High training accuracy with poor dev accuracy is a signature of overfitting
• The cause is limited diversity (~20 designs), not insufficient example count
• Real-world dev sets represent a much broader distribution of car designs
• Training data volume does not substitute for representative diversity

Rubric: Full credit: correctly identifies overfitting to the 20 designs as the cause, distinguishes example quantity from design diversity, and connects this to the mismatch with the real-world dev set distribution. Partial credit: identifies overfitting but does not explain the quantity-versus-diversity distinction.