Learn Before
Why model capacity must be matched with data scale
Question: Explain why a very large neural network by itself is not enough to achieve top performance. What else is needed, and how do the two factors work together?
Sample answer: A very large neural network provides the capacity to represent complicated patterns, but capacity alone does not guarantee strong performance. To use that capacity well, the model also needs a very large amount of training data. The data supplies enough varied examples for the network to learn reliable patterns instead of merely memorizing the training set. In practice, high performance comes from combining a high-capacity model with enough data to train it effectively.
Key points:
- A very large neural network gives high capacity.
- A very large amount of data is also required.
- Strong performance depends on using both together.
Rubric: The response must explicitly mention the need for a very large amount of data together with a large neural network. It should explain that both are needed to achieve the best performance.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Increasing Model Capacity and Training Data as Dependable Improvements
Large Labeled Datasets Can Make Neural Networks a Strong Choice
What setup is most likely to achieve the strongest neural network performance?
Larger models can sometimes achieve better results
Two ingredients behind _____ performance
Matching model scale to performance outcomes
How to pursue better performance with a large model
Why model capacity must be matched with data scale
Expanding a text classification system
What two ingredients are needed to reach peak neural network performance?
What happens when a neural network is made larger?
Model size and data volume