More often than not, there are predictors in our data set that are not associated with the response. Because of this it is useful to narrow down our model to include only a subset of predictors.
- Forward Selection
- Backward Selection
- Mixed Selection


Methods for deciding important variable

Here is an example of multiple linear regression using R:
```r
library(MASS) #  MASS package is a very large collection of data sets and functions
library(ISLR) # ISLR package includes the data sets associated with this book
lm.fit=lm(medv∼lstat+age,data=Boston)# medv as a response, lstat and age as two predictors
summary(lm.fit) # gives us p-values and standard errors for the coeﬃcients, as well as the R2 statistic and F-statistic for the model

#The syntax lm(y∼x1+x2+x3) is used to ﬁt a model with three predictors, #x1, x2, andx3. The summary() function now outputs the regression #coeﬃcients for all the predictors. 

#We can also make a regression using all predictors with the following code:
lm.fit=lm(medv∼.,data=Boston) 
```

Example of Multiple Linear Regression Using R

```R 
#confint() provides information on confidence intervals for the coefficient estimates
#This can be used with both simple and multiple linear regression

confint(lm.fit)
```
Below is an example of the output information you can get by calling confint(lm.fit):

(from Pg. 111 of Introduction to Statistical Learning with Applications in R).

confint() in R with linear regression

Multiple Linear Regression is a linear regression that uses **two or more independent variables** to predict the outcome of a response variable. It is more effective and practical to predict or estimate dependent variables by the optimal combination of multiple independent variables than only one independent variable. 
$$y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + ... + \beta_p x_{ip} + \epsilon$$

University of Michigan - Ann Arbor

Based on the number of independent variables used, there are two types of linear regression:
- Simple Linear Regression
- Multiple Linear Regression

Types of linear regression

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Simple Linear Regression predicts a model for the relationship between a predictor matrix (independent variable) $X$ and the response (dependent variable) $Y$.
$$Y=\beta _1 X+\beta _0+\epsilon$$
The coefficients $\beta _1$ and $\beta _0$ are called slope and intercept, respectively. The estimated values for these coefficients are shown as $\hat{\beta _1}$ and $\hat{\beta _0}$.
$$\hat{Y}=\hat{\beta _1} X+\hat{\beta _0}$$

Simple Linear Regression

Multiple Linear Regression

While simple linear regression can help us understand the relationship between a single predictor and a response, using more than one predictor often proves to be beneficial in predicting responses. Multiple linear regression assigns different slope coefficients to each predictor to build a more comprehensive model than its simple counterpart.

Simple Linear Regression vs. Multiple Linear Regression

https://towardsdatascience.com/regularization-in-machine-learning-76441ddcf99a#:~:text=This%20is%20a%20form%20of,linear%20regression%20looks%20like%20this

Learn Before

Related

Learn After