A correlation between variables indicates a statistical relationship, but it does not provide any evidence about causation, regardless of how strong that relationship may be. Just because two variables move together does not mean that a change in one causes a change in the other. In scientific research, the only definitive way to demonstrate a cause-and-effect relationship is by conducting a controlled experiment.

Correlation Does Not Imply Causation

The value of a correlation coefficient indicates the strength of the relationship between two variables. A coefficient closer to 1 or -1 signifies a strong relationship where changes in one variable are highly predictable from the other. As the coefficient approaches 0, the relationship weakens, and predictability decreases. A coefficient of 0 means the variables are not related, and one cannot be predicted from the other.

Interpreting Correlation Coefficient Values

Depending on the values of other predictors in a multiple linear regression model, the coefficient of a predictor may be positive, indicating an association between an increase in the value of this predictor and an increase in the value of $Y$, or the coefficient can may be negative, indicating an association between a decrease in the value of this predictor and a decrease in the value of $Y$.

University of Michigan - Ann Arbor

1. Is the relation between the predictors $X_1, X_2, X_3, ..., X_P$ and the outcome $Y$ causal?
2. Is the correlation between the predictors $X_1, X_2, X_3, ..., X_P$ and the outcome $Y$ linear or non-linear?
3. Which predictors $X_1, X_2, X_3, ..., X_P$ are correlated, and which ones are not correlated with outcome $Y$?
4. What is the direction of the correlation between each predictor $X_1, X_2, X_3, ..., X_P$ and the outcome $Y$?
5. What is the magnitude of the correlation between each predictor $X_1, X_2, X_3, ..., X_P$ and the outcome $Y$?

Types of inferential questions in statistical learning

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Methods:
- Subset selection
- Shrinkage
- Dimension reduction

Criteria:
(a) Indirectly estimate test error by making adjustment to the training error:
- Adjusted R-Squared
- AIC
- BIC
- Mallow’s $C_p$
(b) Directly estimate the test error:
- Cross-validation

Methods of choosing important predictors to improve interpretability

Direction of correlation between each predictor and outcome

Causal inference refers to making conclusions about finding the reasonings behind the relationship of two or more variables. After coming to some conclusions about statistical data, we ask why such relationships exist between variables.

Learn Before

Related

Learn After