The difference between statistics of independence between input and residual: $$C_{IR} = I(R_y, X) - I(R_x, Y)$$

Input/Residual Independence (Cause-Effect Pair Filter)

The difference between the R-squared statistics measuring goodness of fit in both directions: $$C_{R2} = R_y^2 - R_x^2$$

R-squared (Cause-Effect Pair Filter)

The difference in (discretized) entropy of the marginal distributions: $$C_H = H_x - H_y$$, with $$H_x = \sum_{k=1}^m p_k \log p_k$$ using $$m = 20$$ bins.

Entropy (Cause-Effect Pair Filter)

The difference between the conditional distributions' average standard deviation in both directions: $$C_{CDS} = CDS(X|Y) - CDS(Y|X)$$

Conditional Distribution Standard Deviation (Cause-Effect Pair Filter)

The difference in average slope in both directions: $$C_S = S_y - S_x$$, with $$S_y = \sum_{k=1}^{m-1} p_k \frac{\Delta y_k}{\Delta x_k}$$ using $$m = 20$$ intervals.

Slope (Cause-Effect Pair Filter)

Several cause-effect pair filters (not involving any learning) have been proposed as causation coefficients, based on various data generative assumptions. Here are some examples: - R-squared $$C_{R2}$$ - Input/Residual Independence $$C_{IR}$$ - Entropy $$C_H$$ - Slope $$C_S$$ - Conditional Distribution Standard Deviation $$C_{CDS}$$

University of Michigan - Ann Arbor

Google

Consider a pair of random variables $$(X, Y)$$ with ground truth $$G \in \{X \rightarrow Y, X \leftarrow Y, X \leftrightarrow Y, X \perp Y\}$$ drawn from an unknown mother distribution, and an estimator $$\hat{G}$$ of $$G$$ based on $$(X, Y)$$.

A causation coefficient $$C(X, Y)$$ is defined as a real scalar value where a larger $$C(X, Y)$$ indicates higher confidence that $$G = [X \rightarrow Y]$$. It must have the following properties:
1. [Anti-symmetry] $$C(X, Y) = -C(Y, X)$$
2. [Discriminant] $$C(X, Y) > \theta \ge 0 \Rightarrow \hat{G} = [X \rightarrow Y]$$ (and $$\hat{G} \neq [X \rightarrow Y]$$ otherwise)
3. [Arbitrary units] $$C(aX+b, cY+d) = C(X, Y)$$ for all $$a, b, c, d \in \mathbb{R}$$ where $$a \neq 0$$ and $$c \neq 0$$.

Causation Coefficient

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

The advantages of using a continuous causation coefficient are twofold:
- First, the absolute value $$| C ( X , Y )|$$ represents prediction confidence. When $$| C ( X , Y )|$$ approaches 0, the causal direction is unclear, meaning variables may be independent or dependent due to a confounding effect.
- Second, $$C ( X , Y )$$ can rank pairs of variables, supporting confidence-based scores like precision-recall or area under the ROC curve. This helps practitioners prioritize experiments to assess causal predictions and orient edges in partially oriented causal graphs.

Learn Before

Related

Learn After