This kind of methods range from very simple class of functions with LINGAM to more complex class of functions such as PNL, with each time a trade-off between the identifiability and the generality of the proposed approach as shown in the picture.

Indeed, when the class of function is very restricted, there are fewer non identifiable cases, but the model can only successfully be used when encountering very specific observed data (such as data generated by linear mechanisms). When the class of functions becomes larger, the model becomes more general and can be used for more types of distribution, but at the cost of more non identifiable cases. In the extreme case of a completely general model without restriction on the class of mechanisms, all pairs become non identifiable.

Examples and Reviews of Methods with a Restricted Class of Mechanisms

This class of methods rely on restrictive class of admissible mechanisms $\mathcal{F}$ and focus on identifiability results. The main idea is to show that in some cases there exist $f\in \mathcal{F}$ and $Q_N$ such that the hypothesis $\mathcal{B}_{\mathcal{G}, f, Q_N}$ holds in the causal direction $\mathcal{G}$ with respect to the observed data while there do not exist any $f' \in \mathcal{F}$ nor such $Q'_N$  that $\mathcal{B}_{\mathcal{G'}, f', Q'_N}$ holds in the opposite causal direction $\mathcal{G'}$. 


University of Michigan - Ann Arbor

According to the type of functional involved, their fit scores, their complexity scores and their way to deal with the trade-off fit/complexity, three main families of methods emerge: 
 - Methods with a restricted class of mechanisms. 
 - Non-parametric methods. 
 - Methods exploiting the independence between cause and mechanism.

Main Algorithms for Cause-Effect Inference

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

Methods with a Restricted Class of Mechanisms

Methods with a restricted class of mechanisms always assumed that the causal mechanisms belong to a restricted class of functions $\mathcal{F}$. However, this priori restriction poses serious practical limitations when the task is to infer the causal direction on real data. Indeed in reality the mechanisms are often far from linearity and the interaction between noise and cause may be more complex than additive or even post-nonlinear noise. 

This is why more general methods have been proposed following pioneer works by Stegle et al. These methods in general offer better overall results on real data as they are more flexible, but they come with a loss of theoretical identifiability results, as no explicit restriction on the class of function is imposed. The causal direction is often recovered by setting a smooth prior on the complexity of the mechanisms.

Non-Parametric Methods

This class of methods exploits the notion of complexity of the causal mechanisms from a different perspective. They are based on the principle saying that when X → Y the distribution of the cause $P_X$ should not contain information that can be useful to derive the conditional model $Q_{Y | X}$ on the data.

Learn Before

Related

Learn After