Learn Before
Regularization often lets a larger model remain usable
A bigger model can raise variance and make overfitting more likely. However, when the training procedure includes effective regularization, increasing model size often does not harm test performance much. In that case, the biggest practical cost of using the larger model is usually extra computation and memory, not a large drop in generalization.
0
1
Tags
Data Science
D2L
Dive into Deep Learning @ D2L
Bayesian Statistics
Statistics
Machine Learning
Deep Learning
Supervised Learning
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why does regularization prevent overfitting?
Popular Regularization Techniques in Deep Learning
Human Level Performance: Based on the evidence below, which two of the following four options seem the most promising to try?
Local Constancy and Smoothness Priors
Parameter Sharing
Parameter Tying
L1 regularization and L2 regularization
MTL as a Regularizer
Parameter Penalties in Classical Regularization
Regularization often lets a larger model remain usable
Learn After
What Usually Happens When a Well-Regularized Model Gets Bigger?
True or False: A very large model can still memorize quirks in the training set if regularization is absent or too weak.
The main reason to postpone a larger model is the extra _____.
Match each training setup to its typical effect on variance and overfitting.
Put the model-size and regularization reasoning in order.
How Regularization Changes the Model-Size Tradeoff
Decide what matters most when expanding a well-regularized model.
Why does adding a helpful feature not necessarily hurt a model if variance rises?
Which technique adds a penalty term to discourage very large model weights?
True or False: A larger model can never lower bias.