Large Models and More Data Can Weaken the Tradeoff
In contemporary deep learning, having abundant training examples and very high-capacity networks often makes the usual tension between underfitting and overfitting less restrictive. It is often possible to lower bias by using a larger model and adjusting regularization, while adding more data can often lower variance without materially changing bias.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Large Models and More Data Can Weaken the Tradeoff
What is a common effect of adding regularization to a learning algorithm?
True or False: Increasing a neural network’s width or depth often lowers bias, but it can also make overfitting more likely.
Why Bias and Variance Trade Off
What usually happens when you make a neural network larger?
Regularization usually reduces both bias and variance in a model.
Some model changes reduce _____ while often increasing variance, and other changes do the opposite.
Match each sign or adjustment to its role in the bias-variance tradeoff.
Arrange the steps a practitioner follows when using training and dev errors to choose an improvement strategy.
A model is underfitting its training data. Which change is most likely to help?
Lowering one source of error in a model always lowers every other source of error as well.
Effect of regularization on bias and variance
Match each bias-variance concept to its description in practice
Order the steps for deciding whether to increase capacity or strengthen regularization when a model is not performing well.
How do extra predictors and regularization affect bias and variance?
Effect of model expansion versus regularization on overfitting
How do bias and variance move when a learning system changes?
Learn After
Which change is most likely to lower bias in a modern neural network that is underfitting?
Modern deep learning removes the bias-variance tradeoff entirely.
Reducing variance by adding _____
Match each deep learning strategy with its effect on bias or variance
Order the model-improvement steps when a classifier has both high bias and high variance
Why larger data sets and bigger networks change the bias-variance picture
Improve a transaction-risk classifier
Most direct way to lower variance in deep learning
What most weakens the classic bias-variance tradeoff?
True or False: Making a neural network larger always makes variance much worse.