The given observations $\mathcal{D} = \{(x_i,y_i)\}_{i=1}^n$ of the random variables $X$ and $Y$ are independent and identically distributed drawn from $P_{X,Y}$.

Assumption 1:  Identically and Distributed Samples

In the probelm setting, the time for which the observed data have been collected is not available, so time is not explicitly modeled even though it is assumed that causes precede their effects.

Assumption 2: Time

This is the classical faithfulness assumption used in graphical causal inference, transposed for two variables :"if there is a causal relation between $X$ and$Y$, the two variables are not independent".

Assumption 3: Faithfulness

A selection bias corresponds to an unobserved variable on which the two variables $X$ and $Y$ were implicitly conditioned: $X\rightarrow W \leftarrow Y$

In the problem setting, it is assumed that the sample $\{(x_i,y_i)\}_{i=1}^n$, corresponding to the variables $X$ and $Y$ , was collected without selection bias.


 Assumption 4: Selection Bias

Under this assumption it is assumed that X and Y are not common consequences of the same hidden variables ( X ← Z → Y ). 

This causal sufficiency assumption is made by many methods of the cause-effect pair literature as it allows to considerably simplify the cause-effect pair problem. However there are many realistic cases where there are potential confounding effects that can affect both variables (the most typical examples of confounding factors are the age or the gender in epidemiological studies).

Assumption 5: Causal Sufficiency

A feedback loop appears when both X causes Y and Y cause X : $X \Leftrightarrow Y$. Even if the notion of time is not present, this case may happen for example in cross-sectional studies where data are collected over a certain period of time (e.g. temperature and the amount of sea ice).

We will assume in this problem setting that this case is excluded. Therefore the causal graph between X and Y refers to the literature on Directed Acyclic Graph (DAG) (with only two variables).

Assumption 6: Feedback Loops

This case of dependency with a relation X − Y may arise for example if the two variables are linked by a logical or mathematical formula, and not related by a causal relation. 

As an example, if we write that the productivity P in a firm is equal to the total added value VA of the firm divided by the total number of employees N , the two variables P and  VA are linked by a mathematical expression: $P=\frac{VA}{N}$. By observing VA and knowing N we can immediately deduce P , but the opposite is also true as knowing P and N gives VA , with equivalent mathematical relation, without any notion of an underlying system that could have generated one of the variable from the other. 

We do not consider this case in the problem setting.

Assumption 7: Constraint Relation

In general in the literature, it is assumed that there are no measurement noise. 

Such measurement noise may happen if for example the altitude is not measured precisely, but its noisy version noted $\tilde{Z}$. However the variable temperature T is still a function of the original variable Z that is not corrupted by measurement noise. This problem is related to a cause-effect pair problem between T and $\tilde{Z}$ in presence of a latent hidden variable which is the original Z .

 Assumption 8: Measurement Noise

The orders of magnitude of the variable as well as their physical units are not considered in the problem. In most methods, the variables X and Y are re-scaled with zero mean and unit variance.

Assumption 9: Variable Units

$X$ and $Y$ are two one-dimensional random variables in $R$ with joint distribution $P_{X,Y}$. 

The main assumptions usually involved in the problem setting includes:
 - Assumption 1:  Identically and Distributed Samples
 - Assumption 2: Time
 - Assumption 3: Faithfulness
 - Assumption 4: Selection Bias
 - Assumption 5: Causal Sufficiency
 - Assumption 6: Feedback Loops
 - Assumption 7: Constraint Relation
 - Assumption 8: Measurement Noise
 - Assumption 9: Variable Units

University of Michigan - Ann Arbor

Generative algorithms solve the cause-effect pair problem from a reverse engineering perspective: given the available measurements $\{ (x_i,y_i) \}_{i=1}^n $ of the two variables $X$ and $Y$, the task is to discover the underlying causal process that led the variables to take the values they have.

Such methods allow gaining not only a clue about the causal direction but also information about the mechanism itself, making causal discovery less of a black box decision process.

Generative Models for Cause-Effect Problem

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

Notations and Assumptions

A bivariate structural model noted is a triplet $\mathcal{B}_{\mathcal{G}, f, P_N}$, where $\mathcal{G}$ is the causal graph X → Y or Y → X , $f = ( f_X , f_Y )$ is a couple of possibly nonlinear functions and $( N_X , N_Y )$ are two independent random variables drawn according to continuous distribution $P_N = (P_{N_X}, P_{N_Y})$, such that: 
 - $X := f_X ( N_X )$ and $Y := f_Y ( X , N_Y )$ if X → Y 
 - $Y := f_Y ( N_Y )$ and $X := f_X ( Y , N_X )$ if Y → X

Bivariate Structural Model

The determination of the best explainable model is based in the literature on these main lines: 
 - Different candidate bivariate models (hypotheses) are evaluated in both directions.
 - For each candidate model one evaluates a score monitoring the trade-off between the fit of the model (meaning its adequacy to the observational data) and the complexity of the mechanisms involved. 
 - The model with the best score is returned, with its corresponding causal arrow X → Y or Y → X .

Determination of the Best Explainable Model

According to the type of functional involved, their fit scores, their complexity scores and their way to deal with the trade-off fit/complexity, three main families of methods emerge: 
 - Methods with a restricted class of mechanisms. 
 - Non-parametric methods. 
 - Methods exploiting the independence between cause and mechanism.

Main Algorithms for Cause-Effect Inference

The cause-effect inference problem is relatively new in the Machine Learning literature and there are still a lot of open problems to be addressed in order to build a robust tool that can be used by the practitioner to confirm for example if a given treatment has an impact or not on a given disease or to discover if a gene has a regulatory power on an other one. Here are the open problems:
 - Relax the Causal Sufficiency Assumption
 - Need for Real Datasets of a Big Size
 - Biased Assessment Due to Artifacts in Data
 - Extension of the Generative Approach for Categorical Variables
 - Extension of the Pairwise Settnng for Complete Graph Inference
 - Computational Complexity Limitations
 - Relax Restrictive Assumptions on Causal Mechanisms

Learn Before

Related

Learn After