McElreath, R. (2016). Statistical rethinking: A Bayesian course with examples in R and Stan. Page 152 


Multiple Regression Model

1. Reveal spurious correlation
2. Uncover hidden association

Benefits of Multiple Regression

1. Possible to cause spurious correlation
2. Hide real associations

Shortcomings of Multiple Regression

Q: Does marriage cause divorce?
What are the variables that cause divorce?

Example: Marriage and Divorce

1. Nominate the predictor variables 
2. Make a parameter that will measure the association with the outcome for each predictor 
3. Multiply the parameter with variable and add them together

Steps of Applying Multiple Regression Model

Three good interpretative plot models:
1) Predictor residual plots. These plots show the outcome against residual predictor values. They are useful for understanding the statistical model, but not much else.
2) Posterior prediction plots. These show model-based predictions against raw data, or otherwise display the error in prediction. They are tools for checking fit and assessing predictions. They are not causal tools. 
3) Counterfactual plots. These show the implied predictions for imaginary experiments. These plots allow you to explore the causal implications of manipulating one or more variables.


Plotting multivariate posteriors.

A categorical variable is a type of variable that represents a characteristic or quality, typically assessed by assigning a descriptive category label to each individual rather than a number. Examples include a person's chosen major, nationality, occupation, or whether they are receiving psychotherapy.

Categorical Variable

Multiple regression model looks like the polynomial model which was introduced before but we will dive into it, and use this method to figure out "correlation". 
Multiple regression have more independent varialbles, or explanatory variables, to determine the output (the dependent variable)

University of Michigan - Ann Arbor

Ohio State University, Columbus

Multiple regression is a statistical technique that extends simple regression by measuring several predictor variables ($$X_1, X_2, X_3, \dots, X_i$$) and using them to predict a single outcome variable ($$Y$$). It can also be utilized to describe the complex relationship between the outcome variable and a set of predictors. The result of this analysis is an equation that expresses the outcome variable as an additive combination of all the measured predictor variables.

Multiple Regression

- Uses powers of a variable (squares and cubes) as extra predictors 
- Need to first standardize the predictor variable 
Considerations: 
- A better fit to the sample does not necessarily mean a better model
- Fitting a causal relationship gives us no new information about the model 

Polynomial regression

- Spline: a smooth function built out of smaller, component functions 
- B-splines divide the range of a predictor variable into parts
- Assign a parameter to each part and the parameters are turned on and off so they sum to a fancy curve 
- Create new predictor variables that make up a basis function

Basis-Splines (B-Splines)

Multiple Regression Model 

The multiple regression equation expresses the outcome variable as an additive combination of several predictor variables. Its general form is: $$Y = b_1X_1 + b_2X_2 + b_3X_3 + \dots + b_iX_i$$, where $$Y$$ is the predicted outcome, the $$X$$ terms are the individual predictor variables, and the corresponding $$b$$ terms are their regression weights.

Multiple Regression Formula

The primary advantage of multiple regression is its ability to show whether a specific predictor variable makes a unique contribution to an outcome variable over and above the contributions made by other predictors. This allows researchers to statistically control for other variables while examining a specific relationship.

Advantage of Multiple Regression

If a researcher investigates how income and health relate to happiness, they face a confound because income and health are correlated. By using a multiple regression analysis that includes both as predictor variables, the researcher can statistically control for one while examining the other. This determines whether the part of income unrelated to health predicts happiness, and whether the part of health unrelated to income predicts happiness.

Learn Before

Related

Learn After