Choosing the Weight for Auxiliary Data
When training on a main dataset together with a larger auxiliary dataset, a mixing coefficient such as beta can control how much the auxiliary examples influence the objective. For instance, if 6,000 labeled storefront images are combined with 144,000 web images, a beta value of 1/24 would make their total contributions equal. In practice, beta is then adjusted by checking performance on the development set.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Choosing the Weight for Auxiliary Data
When is it sensible to give auxiliary training examples less influence during training?
Reducing the weight of secondary data can make model training less demanding when that data is not the main target distribution.
Use re-weighting only when the extra data comes from a _____ distribution than the dev/test set.
When to Reduce the Weight of Auxiliary Data
Order the steps for deciding whether to reduce the weight of auxiliary training data.
Why Equal Weighting Can Be Expensive
True or False: Reducing the weight of auxiliary training data is the same as deleting those examples from training.
When mixing a small auxiliary dataset with a main dataset, reducing its _____ can keep the model balanced.
Match each term to its best description when auxiliary data is used for training.
Order the reasoning that shows why reducing the influence of a large auxiliary dataset can make a compact network sufficient.
When to Reduce the Weight of Auxiliary Data
Weighting Target and Auxiliary Data Under Compute Limits
Effect of Lower Weight on Model Capacity
Learn After
Which beta value gives the two image collections equal total weight?
Does beta set how much the auxiliary image set influences the total loss?
Setting beta to _____ makes the two data collections contribute the same total weight.
Match each part of the weighted loss to its role.
Order the steps for setting the auxiliary-data weight.
Explain what beta = 1/40 does when mixing two training pools
Interpreting beta when two data sources differ in size
Why can beta = 1/25 balance collections of very different sizes?
How should you choose a different mixing coefficient?
A tuning constant for auxiliary data must always be set to 1/50.