Learn Before
Feature Selection and the Variance–Bias Trade-off
Dropping inputs can sometimes make a model less sensitive to noise and therefore reduce variance. The trade-off is that removing features can also raise bias if useful information is lost. Small reductions in the feature set often have little effect on bias, but large cuts can matter a lot, especially when important predictors are removed. With large datasets and deep learning, teams often keep a broad set of inputs and let the model learn which ones matter, while feature selection is often more helpful when data is limited.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Regularization Lowers Variance but Raises Bias
Stopping Training When Validation Performance Stops Improving
Feature Selection and the Variance–Bias Trade-off
Using a Smaller Model to Control Variance
What is usually the first choice for reducing variance when enough data are available?
Regularization can lower variance even if it increases bias.
Stopping training early based on _____-set error can reduce variance.
Match each variance-reduction tactic with the main consideration that goes with it.
Order the steps for deciding how to reduce overfitting in a model.
Why should shrinking a model be treated as a cautious option when variance is high?
Prefer regularization when model cost is acceptable
How can error analysis suggest new input features?
Which choice describes early stopping for a model with high variance?
Installing a smaller set of features will always reduce variance without changing bias.
Learn After
How can feature selection change a model's error behavior?
Reducing a model's input set from 1,000 features to 900 features is unlikely to greatly increase bias.
In many data-rich deep learning projects, people often provide _____ features to the learner and let it determine which ones matter most.
Match each feature-reduction case to its likely effect on bias.
Order the steps for deciding whether to drop features to reduce variance.
When is feature selection especially useful?
When training data is abundant for deep learning, teams often rely less on manual feature pruning and more on letting the model learn from a broad set of inputs.
A tenfold reduction in input features
Match each idea to the description that best fits feature selection and variance reduction.
Order the steps for judging whether cutting down the feature set is likely to raise model bias.
Choosing Whether to Drop Predictors
Choosing Feature Reduction for a Sensor Fault Classifier
What Plentiful Data Changes About Feature Selection