The difference between the R-square statistic measuring goodness of fit in both direction: 
$C_{R2} = R_y^2-R_x^2$

R-squared (Cause-Effect Pair Filter)

The difference between statistics of independence between input and residual: 
$C_{IR} = I ( R_y , X ) − I ( R_x , Y ) $

Input/Residual Independence (Cause-Effect Pair Filter)

The difference in (discretized) entropy of the marginal distributions: 
$C_H = H_x − H_y$ , with $H_x = \sum_{k=1}^m p_k \log p_k$
using m = 20 bins

Entropy (Cause-Effect Pair Filter)

The difference in average slope in both direction: 
$C_S = S_y − S_x$ , with $S_y = \sum_{k=1}^{m-1} p_k \Delta y_k/\Delta x_k$ 
using m = 20 intervals

Slope (Cause-Effect Pair Filter)

The difference between conditional distributions average standard deviation, in both directions: 
$C_{CDS} = CDS ( X | Y ) − CDS ( Y | X )$

Conditional Distribution Standard Deviation (Cause-Effect Pair Filter)

Several cause-effect pair filters (not involving any learning) have been proposed as causation coefficients, based on various data generative assumptions. Here are some examples:
 - R-squared $C_{R2}$
 - Input/Residual Independence $C_{IR}$
 - Entropy $C_H$
 - Slope $C_S$
 - Conditional Distribution Standard Deviation $C_{CDS}$

University of Michigan - Ann Arbor

Consider a pair of random variables $( X , Y )$ with ground truth $G ∈\{ X → Y , X ← Y , X \leftrightarrow Y , X ⊥ Y \} $, drawn from an unknown mother distribution. We call an estimator $\hat{G}$ of $G$ based on $( X , Y )$. 

The authors define a causation coefficient as follows:
A causation coefficient $C ( X , Y )$ is a real scalar value, such that the larger $C ( X , Y )$, the more confident we are that $G = X → Y$ . A causation coefficient must have the following desired properties:
1. [Anti-symmetry] 
$C(X,Y) = -C(Y,X)$
2. [Discriminant] 
$C(X,Y)>\theta \leq 0 \Rightarrow \hat{G} = [X\rightarrow Y]$ (and $\hat{G} \neq [X\rightarrow Y]$ otherwise)
3. [Arbitrary units] 
$C(aX+b, cY+d) = C(X,Y); a,b,c,d\in R, a\neq 0, c\neq 0$

Causation Coefficient

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

Examples of Cause-Effect Pair Filters as Causation Coefficients

The advantage of using a continuous causation coefficient is twofold. 
 - On one hand, the absolute value $| C ( X , Y )|$ is interpreted as the confidence of the prediction. When $| C ( X , Y )|$ goes to 0, the causal direction is unclear; variables could be considered as either independent, or dependent because of a confounding effect. 
 - On the other hand, $C ( X , Y )$ is used to rank pairs of variables, supporting the definition of confidence based scores such as the precision-recall score or the area under the ROC curve score. From a practitioner’s viewpoint, $C ( X , Y )$ can be used to prioritize experiments in order to assess causal predictions. Additionally, $C ( X , Y )$ allows practitioners to orient edges in partially oriented causal graphs.

Learn Before

Related

Learn After