Why More Compute Helps Deep Learning Use Large Data
Question: Why has larger computational capacity been an important factor in recent deep learning progress when very large training sets are available?
Sample answer: It lets practitioners train models that are large enough to make effective use of very large datasets.
Key points:
- More compute supports training larger neural networks.
- Larger models can benefit from very large datasets.
Rubric: The answer should state that greater computational capacity enables training networks large enough to exploit very large datasets.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What is the main purpose of increased computational scale in deep learning?
A small gap between training error and training-dev error suggests the model generalizes similarly to unseen data from the same distribution.
Why larger compute can improve deep learning
Match each term to its role in explaining why larger compute can accelerate deep learning gains.
Put the steps in order: why more compute can improve deep learning results
When did it become practical to train neural networks large enough to benefit from very large datasets?
Computing power by itself is enough to explain recent deep learning progress.
Neural Network Size and Data
Match each development factor with the limitation it removes in modern deep learning.
Order the actions a practitioner would take when trying to use compute to benefit from a very large dataset.
How More Compute Helps When Training on Very Large Datasets
Choosing Model and Compute Capacity for a Very Large Dataset
Why More Compute Helps Deep Learning Use Large Data