1. Let $$M_0$$ denote the null model, which contains no predictors.
2. For k = 0, 1, ..., p-1:
(a) Consider all p-k models that augment the predictors in $$M_k$$ with one additional predictor.
(b) Choose the best among these p-k models, and call it $$M_{k+1}$$. Here best is defined as having smallest RSS or highest $$R^2$$.
3. Select a single best model from among $$M_0, ..., M_p$$ using cross-validated prediction error, $$C_p$$ (AIC), BIC, or adjusted $$R^2$$.

University of Michigan - Ann Arbor

Forward stepwise selection begins with a model with no inputs and adds the input that improves the model the greatest one-by-one until all the necessary nodes are in the model.

Forward Stepwise Selection

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

Learn Before

Related