Why can down-weighting auxiliary data reduce the required network size?
Question: Answer in one to three sentences: Why does assigning a lower weight to abundant auxiliary images reduce the need for a massive neural network?
Sample answer: A lower weight reduces the pressure on the model to perform equally well on the auxiliary and target sources. The model therefore needs less capacity to represent both while still benefiting from the additional images.
Key points:
- Lower weighting reduces competition for limited representation capacity.
- The model need not be as massive to handle both sources.
- The auxiliary data is retained rather than discarded.
Rubric: The answer must connect lower auxiliary weight to reduced pressure to model both sources and note that the auxiliary data can still be used.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What should a team with limited model capacity prioritize when auxiliary data greatly outnumbers target data?
Does ample model capacity reduce concern about competition between target and auxiliary data?
With constrained computation, give auxiliary images a much _____ weight.
Match each capacity condition or training choice to its implication.
Order the reasoning for choosing an auxiliary-data weighting strategy.
Explain why model capacity changes the importance of training-data matching.
How should a resource-limited image team use a dominant internet dataset?
Why can down-weighting auxiliary data reduce the required network size?
Which observation most strongly signals that auxiliary-image weighting should be reconsidered?
Must a resource-limited team discard all mismatched auxiliary data?