How do bias and variance move when a learning system changes?
Question: What basic tradeoff appears when changes are made to reduce bias or reduce variance in a learning algorithm?
Sample answer: There is a tradeoff: changes that make the model fit the training data better often increase variance, while changes that make the model less sensitive to noise often increase bias.
Key points:
- Reducing bias can come with higher variance.
- Reducing variance can come with higher bias.
- Improving one side of the tradeoff often worsens the other.
Rubric: The response must explain that lowering bias tends to raise variance, and lowering variance tends to raise bias.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Large Models and More Data Can Weaken the Tradeoff
What is a common effect of adding regularization to a learning algorithm?
True or False: Increasing a neural network’s width or depth often lowers bias, but it can also make overfitting more likely.
Why Bias and Variance Trade Off
What usually happens when you make a neural network larger?
Regularization usually reduces both bias and variance in a model.
Some model changes reduce _____ while often increasing variance, and other changes do the opposite.
Match each sign or adjustment to its role in the bias-variance tradeoff.
Arrange the steps a practitioner follows when using training and dev errors to choose an improvement strategy.
A model is underfitting its training data. Which change is most likely to help?
Lowering one source of error in a model always lowers every other source of error as well.
Effect of regularization on bias and variance
Match each bias-variance concept to its description in practice
Order the steps for deciding whether to increase capacity or strengthen regularization when a model is not performing well.
How do extra predictors and regularization affect bias and variance?
Effect of model expansion versus regularization on overfitting
How do bias and variance move when a learning system changes?