Case context: A team builds a training set by mixing 800 hours of clean speech with background noise taken from 12 specific office spaces, creating 800 distinct hours of noisy audio. No exact clip is reused. Their model scores very well on an internal test set built from the same 12 offices, but it performs badly on recordings from offices not included in training.

Question: What is the most likely reason for the gap in performance, and what lesson should the team draw about judging whether the synthetic noise data is truly diverse?

Sample answer: The model likely overfit to the acoustic patterns of the 12 offices rather than learning speech recognition that works across many kinds of background noise. The fact that each noisy training example was unique does not matter much if all of them came from the same small set of noise sources. Because the internal test set uses the same 12 offices, it hides this weakness, while recordings from new offices reveal it. The team should judge diversity by the variety of underlying noise sources, not just by the number of distinct mixtures, and they should test on offices not seen during training.

Key points:

• Unique mixtures can still come from a small and repetitive set of underlying sources.
• The model can overfit to source-specific noise characteristics.
• A test set drawn from the same sources may fail to expose the problem.
• Evaluation should include unseen noise sources, not only more synthetic combinations.

Rubric: Full credit requires identifying overfitting to the limited set of 12 offices as the cause, explaining why the internal test set did not reveal the issue, and recommending evaluation on unseen noise sources rather than relying on clip uniqueness alone. Partial credit is available for identifying the cause without the evaluation lesson.