Learn Before
Collect Training Examples That Resemble the Hard Development Cases
When the training set and the development set come from different distributions, one practical response is to gather additional training examples that look like the dev examples the model handles poorly. For instance, suppose an image classifier is trained mostly on bright daytime photos, but the dev set contains many dim, rainy, evening shots. In that case, collecting more training images from those lighting and weather conditions can reduce the mismatch.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Recognizing Distribution Mismatch from Error Measurements
Collect Training Examples That Resemble the Hard Development Cases
Compare training and development data properties after a mismatch is found
What best describes a mismatch between training data and evaluation data?
A data mismatch problem means the model performs poorly on both training-like data and the development/test data.
Data mismatch and distribution fit
Match each data-distribution term with its description.
Put the diagnostic workflow for dataset shift in the correct order.
What most directly causes a mismatch between training data and evaluation data?
A voice-to-text system can perform well on the training set and training-dev set but still do poorly on the dev set if the dev data comes from a different distribution.
Training Distribution and Evaluation Distribution
Match each performance pattern to the most likely diagnosis of the model's problem.
Put the steps in order for diagnosing a distribution mismatch from model performance.
Explain what a performance gap across two data distributions suggests
Explain why a voice-command model looks strong on one split but weak on another.
Why is this called data mismatch?
Learn After
Data Mismatch May Not Have a Clear Fix
Synthetic Data That Approximates the Dev Distribution
Are Synthetic Training Examples Representative?
What should you do when training results are strong but validation results drop because the data sources differ?
A data mismatch problem happens when a model does well on training data but performs poorly on a dev set that comes from a different distribution.
Matching the Dev Set Environment
Match each data-mismatch idea to its description.
Order the steps for diagnosing and fixing a data mismatch in a customer-feedback classifier.
What most likely explains the model’s weak performance on the development set in this speech project?
Does training on examples that look more like the dev set always eliminate data mismatch?
When a speech recognizer performs poorly on noisy clips in the dev set, one remedy is to collect more training data that better _____ those difficult examples.
Match each part of a traffic-sign recognition scenario to its role in a data mismatch diagnosis.
Order the steps for deciding whether to collect training data that better matches difficult dev examples.
Using Targeted Data to Reduce Distribution Mismatch
Fix a training-dev distribution gap
What training data change helps with a data mismatch problem?