This class of methods rely on restrictive class of admissible mechanisms $\mathcal{F}$ and focus on identifiability results. The main idea is to show that in some cases there exist $f\in \mathcal{F}$ and $Q_N$ such that the hypothesis $\mathcal{B}_{\mathcal{G}, f, Q_N}$ holds in the causal direction $\mathcal{G}$ with respect to the observed data while there do not exist any $f' \in \mathcal{F}$ nor such $Q'_N$  that $\mathcal{B}_{\mathcal{G'}, f', Q'_N}$ holds in the opposite causal direction $\mathcal{G'}$. 


Methods with a Restricted Class of Mechanisms

Methods with a restricted class of mechanisms always assumed that the causal mechanisms belong to a restricted class of functions $\mathcal{F}$. However, this priori restriction poses serious practical limitations when the task is to infer the causal direction on real data. Indeed in reality the mechanisms are often far from linearity and the interaction between noise and cause may be more complex than additive or even post-nonlinear noise. 

This is why more general methods have been proposed following pioneer works by Stegle et al. These methods in general offer better overall results on real data as they are more flexible, but they come with a loss of theoretical identifiability results, as no explicit restriction on the class of function is imposed. The causal direction is often recovered by setting a smooth prior on the complexity of the mechanisms.

Non-Parametric Methods

This class of methods exploits the notion of complexity of the causal mechanisms from a different perspective. They are based on the principle saying that when X → Y the distribution of the cause $P_X$ should not contain information that can be useful to derive the conditional model $Q_{Y | X}$ on the data.

Methods Exploiting the Independence between Cause and Mechanism

According to the type of functional involved, their fit scores, their complexity scores and their way to deal with the trade-off fit/complexity, three main families of methods emerge: 
 - Methods with a restricted class of mechanisms. 
 - Non-parametric methods. 
 - Methods exploiting the independence between cause and mechanism.

University of Michigan - Ann Arbor

Generative algorithms solve the cause-effect pair problem from a reverse engineering perspective: given the available measurements $\{ (x_i,y_i) \}_{i=1}^n $ of the two variables $X$ and $Y$, the task is to discover the underlying causal process that led the variables to take the values they have.

Such methods allow gaining not only a clue about the causal direction but also information about the mechanism itself, making causal discovery less of a black box decision process.

Generative Models for Cause-Effect Problem

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

$X$ and $Y$ are two one-dimensional random variables in $R$ with joint distribution $P_{X,Y}$. 

The main assumptions usually involved in the problem setting includes:
 - Assumption 1:  Identically and Distributed Samples
 - Assumption 2: Time
 - Assumption 3: Faithfulness
 - Assumption 4: Selection Bias
 - Assumption 5: Causal Sufficiency
 - Assumption 6: Feedback Loops
 - Assumption 7: Constraint Relation
 - Assumption 8: Measurement Noise
 - Assumption 9: Variable Units

Notations and Assumptions

A bivariate structural model noted is a triplet $\mathcal{B}_{\mathcal{G}, f, P_N}$, where $\mathcal{G}$ is the causal graph X → Y or Y → X , $f = ( f_X , f_Y )$ is a couple of possibly nonlinear functions and $( N_X , N_Y )$ are two independent random variables drawn according to continuous distribution $P_N = (P_{N_X}, P_{N_Y})$, such that: 
 - $X := f_X ( N_X )$ and $Y := f_Y ( X , N_Y )$ if X → Y 
 - $Y := f_Y ( N_Y )$ and $X := f_X ( Y , N_X )$ if Y → X

Bivariate Structural Model

The determination of the best explainable model is based in the literature on these main lines: 
 - Different candidate bivariate models (hypotheses) are evaluated in both directions.
 - For each candidate model one evaluates a score monitoring the trade-off between the fit of the model (meaning its adequacy to the observational data) and the complexity of the mechanisms involved. 
 - The model with the best score is returned, with its corresponding causal arrow X → Y or Y → X .

Determination of the Best Explainable Model

Main Algorithms for Cause-Effect Inference

The cause-effect inference problem is relatively new in the Machine Learning literature and there are still a lot of open problems to be addressed in order to build a robust tool that can be used by the practitioner to confirm for example if a given treatment has an impact or not on a given disease or to discover if a gene has a regulatory power on an other one. Here are the open problems:
 - Relax the Causal Sufficiency Assumption
 - Need for Real Datasets of a Big Size
 - Biased Assessment Due to Artifacts in Data
 - Extension of the Generative Approach for Categorical Variables
 - Extension of the Pairwise Settnng for Complete Graph Inference
 - Computational Complexity Limitations
 - Relax Restrictive Assumptions on Causal Mechanisms

Learn Before

Related

Learn After