Using Targeted Data to Reduce Distribution Mismatch
Question: In machine learning, explain how collecting targeted training examples can reduce a data mismatch problem. Illustrate your answer with a speech recognition system.
Sample answer: A data mismatch problem occurs when the examples used for training are noticeably different from the examples the model must handle during development or deployment. One practical response is to collect additional training data that resembles the difficult cases. For example, suppose a speech recognizer is trained mostly on clean headset recordings, but the dev set includes many short utterances captured in a noisy subway station. The model may do well on the training distribution but poorly on those subway clips. If the team deliberately records 300 additional subway-like utterances and adds them to training, the model can learn the acoustic patterns of that environment and channel condition. This narrows the gap between training and dev data and improves performance on the hard cases.
Key points:
- Data mismatch means the training and dev distributions differ.
- A good remedy is to add training examples that resemble the hard dev examples.
- In the speech recognition example, clean headset audio and noisy subway audio represent the mismatch.
- Adding targeted speech data helps the model adapt to the difficult environment.
Rubric: The answer should define data mismatch, explain why targeted data helps, and apply the idea correctly to a speech recognition scenario with different training and dev conditions.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Data Mismatch May Not Have a Clear Fix
Synthetic Data That Approximates the Dev Distribution
Are Synthetic Training Examples Representative?
What should you do when training results are strong but validation results drop because the data sources differ?
A data mismatch problem happens when a model does well on training data but performs poorly on a dev set that comes from a different distribution.
Matching the Dev Set Environment
Match each data-mismatch idea to its description.
Order the steps for diagnosing and fixing a data mismatch in a customer-feedback classifier.
What most likely explains the model’s weak performance on the development set in this speech project?
Does training on examples that look more like the dev set always eliminate data mismatch?
When a speech recognizer performs poorly on noisy clips in the dev set, one remedy is to collect more training data that better _____ those difficult examples.
Match each part of a traffic-sign recognition scenario to its role in a data mismatch diagnosis.
Order the steps for deciding whether to collect training data that better matches difficult dev examples.
Using Targeted Data to Reduce Distribution Mismatch
Fix a training-dev distribution gap
What training data change helps with a data mismatch problem?