Auxiliary Data Can Strain Limited Model Capacity
If an extra training set is much larger than the target-domain set, a model with limited capacity may have trouble fitting both sources well at the same time. Reducing the weight of the auxiliary examples can let the model use them without needing an excessively large network.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Auxiliary Data Can Strain Limited Model Capacity
Why can adding studio photos hurt recognition of handheld-device photos?
Features tied to one data source can consume capacity needed for another data source.
Domain-specific patterns can consume a model's _____ capacity.
Match each concept to its role in a capacity tradeoff.
Order the reasoning from added studio photos to possible performance harm.
Why source differences can reduce model capacity for the target domain
Assess the risk of mixing website photos into a phone-photo model
How can source-domain details consume a model’s capacity?
Which difference between two data sources most directly creates the risk of wasted model capacity?
Heavy reliance on source-only patterns can reduce deployment accuracy.
Learn After
When a small model is trained with a tiny target dataset and a much larger auxiliary dataset, what should the team focus on first?
A model with enough capacity can often learn patterns from both a main dataset and a smaller auxiliary dataset without necessarily running out of room.
With limited compute, assign extra context a much _____ weight.
Match Each Model Setting to Its Likely Effect
Order the reasoning for selecting a weighting strategy for extra training data.
How model size affects the importance of training-set alignment
How should a small vision team use a much larger auxiliary image collection?
Why does reducing the influence of extra training data ease model-size demands?
Which situation most strongly suggests lowering the weight of a large auxiliary data source?
Should a small team always ignore extra training data that comes from a different distribution?