A theoretical identifiability has been derived by Shimizu et al. who prove that if $P_{X , Y}$ admits the linear model $Y := aX + N_Y$ with $X\perp \! \! \! \perp N_Y$  (model 1), then there exist $b\in R$ and a random variable $N_X$ such that $X := bY + N_X$ with $Y\perp \! \! \! \perp N_X$ (model 2) if and only if $X$ and $N_Y$ are Gaussian.

In different words there is only one non-identifiable case corresponding to the linear Gaussian case. Moreover if $X$ or $N_Y$ is non-Gaussian, when the candidate linear model with orientation X → Y holds, the candidate linear model with orientation Y → X does not hold. 

University of Michigan - Ann Arbor

The LiNGAM method was first developed for directed acyclic graph orientation for more than two variables. LiNGAM handles linear structural equation models, where each variable is continuous and modeled as:
$X_i := \sum_{k} \alpha_k P_a^k (X_i) + E_i, i \in [[1,n]]$
with the k th parent of $X_i$ and $α_k$ a real value. Assuming further that all probability distributions of source nodes in the causal graph are non-Gaussian, it is shown that the causal structure is fully identifiable (all edges can be oriented)

Pairwise LiNGAM Inference

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

In the bivariate case, the authors assume that the variables X and Y are non Gaussian, as well as standardized to zero mean and unit variance. The goal is to distinguish between candidate linear causal models. 
The first is denoted by X → Y and defined as:
$Y:=\rho X+N_Y$ with $X \perp \!\!\! \perp N_Y$
The second model with orientation Y → X is defined as: 
$X:=\rho Y+N_X$ with $Y \perp \!\!\! \perp N_X$
The parameter ρ is the same in the two models because it is equal to the correlation coefficient between X and Y

LiNGAM Model

LiNGAM Identifiability Result

The candidate models correspond to a restrictive class of mechanism and the comparison of the candidate models is based on comparison of fit scores at fixed complexity. The fit score used for comparison is the likelihood score derived by Hyvärinen and Smith in this case as: 
$S_{X\rightarrow Y} = -[\sum_{i=1}^n G_X(x^i)+G_{N_Y}(\frac{y^i - \rho x^i}{\sqrt{1-\rho^2}})]+n\log (1-\rho^2) $
where $G_X ( u ) = \log P_X ( u )$ and $G_{N_Y}$ is the standardized log probability distribution function of the residual when regressing Y on X . $S_{Y → X}$ is computed similarly and the causal direction is decided by comparing $S_{X → Y}$ with $S_{Y → X}$.

Learn Before

Related