Why does adding a helpful feature not necessarily hurt a model if variance rises?
Question: In one to three sentences, explain what should be done if adding a new feature makes a model more variable.
Sample answer: If the extra feature increases variance, the recommended response is to add regularization. Regularization usually offsets that extra variance, so the model can still improve overall.
Key points:
- Extra features can raise variance.
- Regularization is the recommended response.
- Regularization usually removes the added variance.
Rubric: Full credit for saying that regularization should be used and that it usually cancels the variance increase. Partial credit for mentioning regularization without explaining the expected effect.
0
1
Tags
Data Science
D2L
Dive into Deep Learning @ D2L
Bayesian Statistics
Statistics
Machine Learning
Deep Learning
Supervised Learning
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What Usually Happens When a Well-Regularized Model Gets Bigger?
True or False: A very large model can still memorize quirks in the training set if regularization is absent or too weak.
The main reason to postpone a larger model is the extra _____.
Match each training setup to its typical effect on variance and overfitting.
Put the model-size and regularization reasoning in order.
How Regularization Changes the Model-Size Tradeoff
Decide what matters most when expanding a well-regularized model.
Why does adding a helpful feature not necessarily hurt a model if variance rises?
Which technique adds a penalty term to discourage very large model weights?
True or False: A larger model can never lower bias.