When there is a confounder $$Z$$ for variables $$X$$ and $$Y$$, plotting all three variables forms a cloud of points in XYZ-space. The analogue of a regression line is a regression plane with the equation $$Y = aX + bZ + c$$. The coefficient $$a$$ is the regression coefficient of $$Y$$ on $$X$$ already adjusted for $$Z$$. It is called a partial regression coefficient and is written as $$r_{YX.Z}$$. If $$Z$$ is the only confounder, then $$a$$ represents the average causal effect of $$X$$ on $$Y$$.

Partial Regression Coefficient

In the regression model $Y \sim X$, to control for the effect of confounder $Z$ (where $X \leftarrow Z \rightarrow Y$) we can use the regression model $Y \sim X + Z$, i.e., taking the average effect of $X$ on $Y$ across differnt values of $Z$.

University of Michigan - Ann Arbor

Given two variables X, Y and a confounder Z (i.e. X<-Z->Y). To obtain the relationship between X and Y not affected by Z, we can separately compare X and Y in every groups of Z. Then we can take an average of the effects, weighing each group according to its percentage. This method of compensation is called "adjusting for Z" or "controlling for Z" by statisticians.

Deconfounding/Adjusting/Controlling a Measurable Confounder

A regression model predicts a value $$Y$$ given a set of $$X$$ variables. In the regression model $$Y \sim X$$, we measure the effect of $$X$$ on $$Y$$, i.e., we measure the association between changes in values of $$X$$ and changes in values of $$Y$$. Analyzing a regression model is a method used to solve a regression problem.

Reference: James, G., Witten, D., Hastie, T., & Tibshirani, R. (2017). An introduction to statistical learning: with applications in R. New York: Springer.

Regression Model

Pearl, J., & Mackenzie, D. (2018). $$\mathit{The \ Book \ of \ Why: \ The \ New \ Science \ of \ Cause \ and \ Effect.}$$ Basic Books.

The Book of Why

The statistical procedure of controlling for everthing that can be measured is misguided. 

For instance, in a collider $A\rightarrow B \leftarrow C$ where the path between $A$ and $C$ is initially independent, controlling for B would make them dependnet due to the explain-away effect. This means that a back-door is opened.

Also, controlling for descendents of the varibles we are interested in should also be avoided. This is like "partially" controlling for the interested variable itself. 

Controlling for everything is misguided

Deconfounding/Adjusting/Controlling a Measurable Confounder in a Regression Model

- Mean Squared Error(MSE)
- Root-Mean-Squared-Error(RMSE).
- Mean-Absolute-Error(MAE).
- R² or Coefficient of Determination.
- Adjusted R²

Learn Before

Related

Learn After