How More Compute Helps When Training on Very Large Datasets
Question: Explain why increased computational capacity can accelerate progress in deep learning, especially when the training data set is very large.
Sample answer: More computational capacity makes it possible to train larger neural networks or train models for longer. That matters because very large data sets often contain patterns that smaller models cannot capture well. When compute is limited, a model may not be big enough to make full use of the available data. With enough compute, the model can scale to a size that can better learn from the large data set.
Key points:
- More computational capacity supports training larger models.
- Larger models can better use very large data sets.
- Without enough compute, the model may be too small to benefit fully from the data.
Rubric: The response must explain that extra compute enables larger neural networks and that those larger networks are needed to take advantage of very large data sets.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What is the main purpose of increased computational scale in deep learning?
A small gap between training error and training-dev error suggests the model generalizes similarly to unseen data from the same distribution.
Why larger compute can improve deep learning
Match each term to its role in explaining why larger compute can accelerate deep learning gains.
Put the steps in order: why more compute can improve deep learning results
When did it become practical to train neural networks large enough to benefit from very large datasets?
Computing power by itself is enough to explain recent deep learning progress.
Neural Network Size and Data
Match each development factor with the limitation it removes in modern deep learning.
Order the actions a practitioner would take when trying to use compute to benefit from a very large dataset.
How More Compute Helps When Training on Very Large Datasets
Choosing Model and Compute Capacity for a Very Large Dataset
Why More Compute Helps Deep Learning Use Large Data