Learn Before
Using a Smaller Model to Control Variance
Reducing the number of layers or units can make a model less sensitive to the training set, but it also lowers capacity and can increase bias. When extra computation is acceptable, regularization is usually the better first choice because it often improves classification results without forcing the model to be smaller. A compact model is most useful when faster training or lower inference cost matters.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Regularization Lowers Variance but Raises Bias
Stopping Training When Validation Performance Stops Improving
Feature Selection and the Variance–Bias Trade-off
Using a Smaller Model to Control Variance
What is usually the first choice for reducing variance when enough data are available?
Regularization can lower variance even if it increases bias.
Stopping training early based on _____-set error can reduce variance.
Match each variance-reduction tactic with the main consideration that goes with it.
Order the steps for deciding how to reduce overfitting in a model.
Why should shrinking a model be treated as a cautious option when variance is high?
Prefer regularization when model cost is acceptable
How can error analysis suggest new input features?
Which choice describes early stopping for a model with high variance?
Installing a smaller set of features will always reduce variance without changing bias.
Learn After
If you want to reduce variance and extra computation is acceptable, which adjustment is usually preferred over shrinking the model?
Reducing a model’s size is usually the first choice for lowering variance when other options are available.
When trying to reduce variance, adding _____ is usually a better first step than simply shrinking the model.
Match each variance-control idea with its best description.
Arrange the steps for handling a model that appears to have high variance.
Why might a practitioner reduce model size even when another approach would better address variance?
Model Capacity and Variance
A smaller model is most useful when _____ training is the main goal, rather than only trying to lower variance.
Match each goal or situation to the technique or consequence most consistent with the guidance on model size and regularization.
Arrange the reasoning used to decide whether shrinking a model is a good response to high variance.
When Should You Shrink a Model Instead of Regularizing It?
Choosing a Variance Reduction Strategy for a Speech Model with Slow Training
When a Smaller Model Is a Reasonable Variance Fix