Training Data Matching Matters More When Model Capacity Is Limited
When additional data is much more numerous than target-distribution data, limited computational resources can make it expensive to model both sources well. Down-weighting the auxiliary source can reduce the need for a very large neural network while still using the additional data.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Training Data Matching Matters More When Model Capacity Is Limited
Why can adding internet images hurt recognition of mobile-app images?
Internet-specific properties may use capacity needed for mobile-app recognition.
Internet-specific properties can consume the network's _____ capacity.
Match each concept to its role in the capacity problem.
Order the reasoning from added internet images to possible performance harm.
Explain the capacity tradeoff created by source-specific image properties.
Diagnose why extra internet images might weaken a mobile-app recognizer.
What does it mean for internet-image properties to “use up” capacity?
Which image-source difference most directly creates the stated capacity risk?
Large source differences can theoretically reduce target-task performance.
Learn After
What should a team with limited model capacity prioritize when auxiliary data greatly outnumbers target data?
Does ample model capacity reduce concern about competition between target and auxiliary data?
With constrained computation, give auxiliary images a much _____ weight.
Match each capacity condition or training choice to its implication.
Order the reasoning for choosing an auxiliary-data weighting strategy.
Explain why model capacity changes the importance of training-data matching.
How should a resource-limited image team use a dominant internet dataset?
Why can down-weighting auxiliary data reduce the required network size?
Which observation most strongly signals that auxiliary-image weighting should be reconsidered?
Must a resource-limited team discard all mismatched auxiliary data?