Indeed in the literature, we distinguish mainly additive noise interaction of the form $y:=\hat{f}_Y(X) + N_Y$, or complex noise interaction of the form $Y:= \hat{f}_Y(X,N_Y)$, where the cause variable and the noise are mixed with a non-linear mechanism.”


University of Michigan - Ann Arbor

After having defined the overall structure, one needs to model the mechanisms, there are two characterizations that need to be specified:
 - Type of interaction between the noise variable and the cause.
 - The class of functions used to define $\hat{f}$.

Mechanisms

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

Type of Interaction between the Noise Variable and the Cause

It may range from linear mechanisms as in LiNGAM algorithm to complex non-linear mechanisms modeled with Gaussian processes or neural networks. 

In general, the more complex the mechanisms are, the more the candidate model can fit the data and the more the model is general (meaning that it can be applied to a wide variety of cases). However, it may result in more difficulty to assess the causal orientation because the candidate model has more chances to fit equally well the data in both directions. 

The Class of Functions Used to Define $\hat{f}$

Now we have all the ingredients required to sample data points of the estimated distribution $Q_{X , Y}$ with the generative model by proceeding as follow: 

1. Draw $\{(n_{X,j}, n_{Y,j})\}_{j=1}^n$ , n samples independent and identically distributed from the joint distribution $Q_{N_X}(\theta_{N_X})\times Q_{N_Y}(\theta_{N_Y})$ of independent noise variables $( N_X , N_Y )$. 
2. Generate n samples $\hat{\mathcal{D}} = \{(\hat{x}_j, \hat{y}_j) \}_{j=1}^n$, where each estimated sample $\hat{x}_j$ of variable $X$ is computed from $\hat{f}_X(\theta_X)$ with the j-th estimated samples $n_{X , j}$; then each estimated sample $\hat{y}_j$ of variable $Y$ is computed from $\hat{f}_Y(\theta_Y)$ with the j-th estimated samples $n_{Y , j}$ and $\hat{x}_j$. 

Generative candidate models supports both interventions and counterfactual reasoning.

Learn Before

Related