Learn Before
Scaling a speech recognition system
Case context: A team has built a speech recognition system using a moderately sized neural network and a dataset of 10,000 hours of audio. The performance has plateaued. They have the budget to either significantly increase the size of their neural network or collect 100,000 more hours of audio, but they cannot do both immediately.
Question: Based on the principle that the best performance comes from having both a very large network and huge data, what should the team diagnose about their ultimate goal, even if they must take steps sequentially?
Sample answer: The team should diagnose that achieving the absolute 'best performance' will ultimately require them to secure both the very large neural network and the huge amount of data. Scaling only one dimension will have limits, so their long-term roadmap must account for expanding both the network capacity and the dataset volume.
Key points:
- Diagnose that scaling only one factor has limits.
- Recognize the ultimate goal is scaling both dimensions.
- Best performance requires a very large network AND huge data.
Rubric: Evaluates if the learner understands that scaling both dimensions is ultimately necessary for the best performance.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Bigger Networks and More Data as Reliable Improvement Levers
One Million Examples Can Favor a Neural Network
Achieving best performance with neural networks
Impact of network size on performance
The two keys to _____ performance
Matching scale components to their effects
Steps to maximize neural network performance
Analyzing the relationship between model size and data volume
Scaling a speech recognition system
The two requirements for top performance
The effect of training larger neural networks
Data requirements for large networks