When there is a confounder $$Z$$ for variables $$X$$ and $$Y$$, plotting all three variables forms a cloud of points in XYZ-space. The analogue of a regression line is a regression plane with the equation $$Y = aX + bZ + c$$. The coefficient $$a$$ is the regression coefficient of $$Y$$ on $$X$$ already adjusted for $$Z$$. It is called a partial regression coefficient and is written as $$r_{YX.Z}$$. If $$Z$$ is the only confounder, then $$a$$ represents the average causal effect of $$X$$ on $$Y$$.

Partial Regression Coefficient

In the regression model $Y \sim X$, to control for the effect of confounder $Z$ (where $X \leftarrow Z \rightarrow Y$) we can use the regression model $Y \sim X + Z$, i.e., taking the average effect of $X$ on $Y$ across differnt values of $Z$.

University of Michigan - Ann Arbor

Given two variables X, Y and a confounder Z (i.e. X<-Z->Y). To obtain the relationship between X and Y not affected by Z, we can separately compare X and Y in every groups of Z. Then we can take an average of the effects, weighing each group according to its percentage. This method of compensation is called "adjusting for Z" or "controlling for Z" by statisticians.

Deconfounding/Adjusting/Controlling a Measurable Confounder

A regression model predicts a value $$Y$$ given a set of $$X$$ variables. In the regression model $$Y \sim X$$, we measure the effect of $$X$$ on $$Y$$, i.e., we measure the association between changes in values of $$X$$ and changes in values of $$Y$$. Analyzing a regression model is a method used to solve a regression problem.

Reference: James, G., Witten, D., Hastie, T., & Tibshirani, R. (2017). An introduction to statistical learning: with applications in R. New York: Springer.

Regression Model

Pearl, J., & Mackenzie, D. (2018). $$\mathit{The \ Book \ of \ Why: \ The \ New \ Science \ of \ Cause \ and \ Effect.}$$ Basic Books.

The Book of Why

Deconfounding/Adjusting/Controlling a Measurable Confounder in a Regression Model

The statistical procedure of controlling for everything that can be measured is misguided. For instance, in a collider $$A \rightarrow B \leftarrow C$$ where the path between $$A$$ and $$C$$ is initially independent, controlling for $$B$$ would make them dependent due to the explain-away effect. This means that a back-door path is opened. Furthermore, controlling for descendants of the variables we are interested in should also be avoided. This is equivalent to "partially" controlling for the variable of interest itself.

Controlling for Everything is Misguided

Regression model evaluation metrics are quantitative measures used to assess how well a model's predictions match the actual observed data. Common metrics include Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), the Coefficient of Determination ($$R^2$$), and Adjusted $$R^2$$.

Learn Before

Related

Learn After