Now we have all the ingredients required to sample data points of the estimated distribution $Q_{X , Y}$ with the generative model by proceeding as follow: 

1. Draw $\{(n_{X,j}, n_{Y,j})\}_{j=1}^n$ , n samples independent and identically distributed from the joint distribution $Q_{N_X}(\theta_{N_X})\times Q_{N_Y}(\theta_{N_Y})$ of independent noise variables $( N_X , N_Y )$. 
2. Generate n samples $\hat{\mathcal{D}} = \{(\hat{x}_j, \hat{y}_j) \}_{j=1}^n$, where each estimated sample $\hat{x}_j$ of variable $X$ is computed from $\hat{f}_X(\theta_X)$ with the j-th estimated samples $n_{X , j}$; then each estimated sample $\hat{y}_j$ of variable $Y$ is computed from $\hat{f}_Y(\theta_Y)$ with the j-th estimated samples $n_{Y , j}$ and $\hat{x}_j$. 

Generative candidate models supports both interventions and counterfactual reasoning.

Sampling Data Points with a Candidate Model

The class of functions used to define the estimated mechanism $$\hat{f}$$ may range from linear mechanisms, as in the LiNGAM algorithm, to complex non-linear mechanisms modeled with Gaussian processes or neural networks. In general, more complex mechanisms allow the candidate generative model to fit the data better and apply to a wider variety of cases. However, increased complexity can make it more difficult to assess causal orientation, as the model has a higher chance of fitting the data equally well in both directions.

Class of Functions Used to Define $$\hat{f}$$

In causal modeling, the interaction between the noise variable and the cause is primarily categorized in two ways:

- **Additive noise interaction**: $$Y := \hat{f}_Y(X) + N_Y$$
- **Complex noise interaction**: $$Y := \hat{f}_Y(X, N_Y)$$, where the cause variable and the noise are mixed through a non-linear mechanism.

Types of Interaction between the Noise Variable and the Cause

When modeling causal mechanisms within a candidate bivariate generative model, two primary characterizations must be specified:

1. The type of interaction between the noise variable and the cause.
2. The class of functions used to define the estimated mechanism $$\hat{f}$$.

University of Michigan - Ann Arbor

Google

To model a continuous underlying bivariate generative process $$\mathcal{B}_{\mathcal{G}, f, P_N}$$ that is assumed to have generated observational data, a candidate bivariate generative model $$\hat{\mathcal{B}}_{\hat{\mathcal{G}}, \hat{f}, Q_N}$$ can be constructed. This candidate model is described by: - A structure defined by a causal orientation $$X \rightarrow Y$$ or $$Y \rightarrow X$$. - Estimated mechanisms modeled by $$\hat{f}$$. - An estimated distribution of noise $$Q_N$$.

Candidate Bivariate Generative Model

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

The authors note $θ_X$ and $θ_Y$ the vectors of parameters of the estimated mechanisms $\hat{f}_X$ and $\hat{f}_Y$. Note respectively $\theta_{N_X}$ and $\theta_{N_Y}$ the vectors of parameters of the distribution of the modeled noise variables $Q_{N_X}$ and $Q_{N_Y}$. The noise variables are independent. The global vector of parameters of the model is noted $\theta = (\theta_X, \theta_Y, \theta_{N_X}, \theta_{N_Y})$ . 

When $\mathcal{G} = X\rightarrow Y$, this candidate generative model (depicted on the figure) generates a distribution $Q_{X,Y}(\theta) = Q_X(\theta_X, \theta_{N_X})Q_{Y|X}(\theta_Y, \theta_{N_Y})$. 

When $\mathcal{G} =Y\rightarrow X$, this candidate model generates a distribution $Q_{X,Y}(\theta) = Q_Y(\theta_Y, \theta_{N_Y})Q_{X|Y}(\theta_X, \theta_{N_X})$. 

Learn Before

Related

Learn After