The Post-NonLinear (PNL) model is evaluated using an independence score between the cause and the noise. As $$\hat{g}_Y$$ is assumed to be invertible, the noise variable can be recovered from the joint distribution $$P_{X, Y}$$ as: $$\hat{N}_Y = \hat{g}_Y^{-1}(Y) - \hat{f}_Y(X)$$. The noise variable is then estimated by functions $$l_1$$ and $$l_2$$ such as $$\hat{N}_Y = l_1(Y) - l_2(X)$$ with $$\hat{N}_Y$$ independent of $$X$$. This solves a constrained nonlinear Independent Component Analysis (ICA) problem, which can be achieved by minimizing $$I(X,\hat{N}_Y; \theta)$$, the mutual information between $$X$$ and $$\hat{N}_Y$$ with respect to the model parameter $$\theta$$. Symmetrically, an optimization of $$I(Y,\hat{N}_X; \theta)$$ is performed. The causal direction $$X \rightarrow Y$$ is preferred if $$I(X,\hat{N}_Y; \hat{\theta}) < I(Y,\hat{N}_X; \hat{\theta})$$, and $$Y \rightarrow X$$ otherwise.

PNL Practical Evaluation

Zhang and Hyvärinen proved that the Post-NonLinear (PNL) model is generally identifiable. If a joint distribution $$P_{X, Y}$$ satisfies a PNL model with orientation $$X \rightarrow Y$$, then $$P_{X, Y}$$ cannot satisfy a PNL model with orientation $$Y \rightarrow X$$, except under specific conditions. The set of non-identifiable distributions $$P_{X, Y}$$ is larger for PNL than for the Additive Noise Model (ANM), but PNL is more general and handles a wider variety of observed distributions.

PNL Identifiability Result

The Post-NonLinear (PNL) model is a generalization of the Additive Noise Model (ANM) that takes into account nonlinear interactions between the cause and the noise, as proposed by Zhang and Hyvärinen. A bivariate Post-NonLinear Model (PNL) $$X \rightarrow Y$$ is defined as $$Y := \hat{g}_Y (\hat{f}_Y(X) + N_Y)$$ with $$X \perp \! \! \! \perp N_Y$$, where $$\hat{f}_Y: \mathbb{R} \rightarrow \mathbb{R}$$ and $$\hat{g}_Y: \mathbb{R} \rightarrow \mathbb{R}$$ are Borel measurable functions, and $$\hat{g}_Y$$ is assumed to be invertible.

University of Michigan - Ann Arbor

Google

This kind of methods range from very simple class of functions with LINGAM to more complex class of functions such as PNL, with each time a trade-off between the identifiability and the generality of the proposed approach as shown in the picture.

Indeed, when the class of function is very restricted, there are fewer non identifiable cases, but the model can only successfully be used when encountering very specific observed data (such as data generated by linear mechanisms). When the class of functions becomes larger, the model becomes more general and can be used for more types of distribution, but at the cost of more non identifiable cases. In the extreme case of a completely general model without restriction on the class of mechanisms, all pairs become non identifiable.

Examples and Reviews of Methods with a Restricted Class of Mechanisms

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

Additive Noise Model is an extension of LiNGAM model to deal with nonlinear mechanism derived by Hoyer et al. 
A bivariate additive noise model (ANM) X → Y is defined as: 
$Y:=f_Y(X) + N_Y$ with $X \perp \! \! \! \perp N_Y$
where $f_Y : \mathbb{R}\rightarrow \mathbb{R}$ is a Borel measurable function.

Additive Noise Model (ANM)

The Linear Non-Gaussian Acyclic Model (LiNGAM) was originally developed for directed acyclic graph orientation involving more than two variables. LiNGAM handles linear structural equation models, where each continuous variable is modeled as: $$X_i := \sum_{k} \alpha_k P_a^k (X_i) + E_i, i \in [[1,n]]$$ with the $$k$$-th parent of $$X_i$$ and $$\alpha_k$$ as a real value. Assuming further that all probability distributions of source nodes in the causal graph are non-Gaussian, it is shown that the causal structure is fully identifiable (all edges can be uniquely oriented).

Learn Before

Related

Learn After