Case context: A team builds a synthetic training set by combining 1,000 hours of clean speech recordings with car noise audio recorded from 10 different cars, producing 1,000 unique hours of noisy audio (no single clip is repeated). The resulting model performs very well on their internal test set, which also uses noise recorded from the same 10 cars, but performs poorly on real-world audio recorded from other cars.

Question: Based on the source concept, what is the most likely cause of this generalization gap, and what does it suggest about how the team should evaluate 'uniqueness' in synthetic noise data?

Sample answer: The most likely cause is that the algorithm overfit to the acoustic characteristics of the 10 specific cars used to generate the noise, even though each hour of noise audio was technically unique (no clip was literally repeated). The internal test set shares the same 10 cars as the training set, so it does not reveal this overfitting, but real-world audio from different cars exposes it. This suggests the team should not treat 'uniqueness' of noise clips as sufficient; they should instead evaluate the diversity of the underlying noise sources (e.g., number of distinct cars) and test on audio from cars not represented in training.

Key points:

• 1,000 unique hours of noise from only 10 cars can still cause overfitting to those cars
• Uniqueness of individual clips does not guarantee diversity of underlying sources
• An internal test set drawn from the same limited sources will not reveal this overfitting
• Real-world audio from different cars exposes the generalization gap
• Evaluating synthetic data should consider diversity of sources, not just quantity or clip uniqueness

Rubric: Full credit identifies overfitting to the limited set of 10 cars as the cause, explains why the internal test set failed to reveal it, and recommends evaluating source diversity (not just clip uniqueness) and testing on unseen sources. Partial credit for identifying the cause without the evaluation recommendation.