When to Reduce the Influence of Extra Training Data
If a secondary dataset comes from a noticeably different pattern than the development and test data, or if it contains far more examples than the main training set, you can assign those examples a smaller loss weight. This keeps the model focused on the target data while still learning useful signals from the extra source, and it may make training easier for a smaller network.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Do Not Randomly Mix Sources When Building Validation and Test Sets
Use a Small Amount of Target-Distribution Data in Training with Auxiliary Data
When to Reduce the Influence of Extra Training Data
Training-Development Set
Comparing Errors Across Two Data Sources
Training–Dev Distribution Gap
Domain adaptation solves only certain distribution-shift problems
Choosing Development and Test Data for a Specific Application
Factory Photos and Inspection Tablet Images as a Distribution Mismatch Example
Random Train/Test Splits Can Be Misleading When Deployment Data Differs
Which data should define the dev and test sets for the plant-disease app?
Development and test examples should be drawn to match the kind of data the deployed system is expected to see.
Development and test sets should match the kind of data you expect later.
Match each data group for the plant-disease app with its role.
Order the dataset choices when training data and evaluation data come from different sources.
Why training data and evaluation data may come from different sources
Find the evaluation-set mismatch in a voice-command detector.
Why did strong desktop-photo results not predict mobile-upload success?
Choosing Training and Evaluation Data from Two Image Sources
If training uses extra web-sourced photos, the dev and test sets must also contain web-sourced photos.
Learn After
Choosing the Weight for Auxiliary Data
When is it sensible to give auxiliary training examples less influence during training?
Reducing the weight of secondary data can make model training less demanding when that data is not the main target distribution.
Use re-weighting only when the extra data comes from a _____ distribution than the dev/test set.
When to Reduce the Weight of Auxiliary Data
Order the steps for deciding whether to reduce the weight of auxiliary training data.
Why Equal Weighting Can Be Expensive
True or False: Reducing the weight of auxiliary training data is the same as deleting those examples from training.
When mixing a small auxiliary dataset with a main dataset, reducing its _____ can keep the model balanced.
Match each term to its best description when auxiliary data is used for training.
Order the reasoning that shows why reducing the influence of a large auxiliary dataset can make a compact network sufficient.
When to Reduce the Weight of Auxiliary Data
Weighting Target and Auxiliary Data Under Compute Limits
Effect of Lower Weight on Model Capacity