An open problem concerns the fact that all these methods rely on specific assumptions on the underlying data generative process. All of them can work well when these specific assumptions are encountered in the observed data. This is why a new Machine Learning approach has appeared in recent years. It is based on the idea to combine all the successful algorithms into a single meta algorithm that could benefit from the advantages of each of them.

University of Michigan - Ann Arbor

The cause-effect inference problem is relatively new in the Machine Learning literature and there are still a lot of open problems to be addressed in order to build a robust tool that can be used by the practitioner to confirm for example if a given treatment has an impact or not on a given disease or to discover if a gene has a regulatory power on an other one. Here are the open problems:
 - Relax the Causal Sufficiency Assumption
 - Need for Real Datasets of a Big Size
 - Biased Assessment Due to Artifacts in Data
 - Extension of the Generative Approach for Categorical Variables
 - Extension of the Pairwise Settnng for Complete Graph Inference
 - Computational Complexity Limitations
 - Relax Restrictive Assumptions on Causal Mechanisms

Open Problems and Extensions

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

To build such useful tools for practitioners, one of the first assumption that needs to be relaxed is the causal sufficiency assumption as in a lot of real problems it is very rare to find a cause-effect problem that is not affected by hidden common confounder that can affect both variables such as age or gender. 

One idea proposed in the literature is to model confounders by introducing correlation between the noise variables $N_X$ and $N_Y$ that affect $X$ and $Y$ as in or by modelling all the unobserved confounding effects by a new noise variable $N_{ XY}$ entering in the generation process of $X$ and $Y$.

Relax the Causal Sufficiency Assumption

An important problem that is often overlooked in the “cause-effect pair community” concerns benchmarking. The main real data benchmark used by authors to compare the different methods by the community is the Tübingen Dataset, composed of only 100 hundred pairs which are often very similar. It is indeed very difficult to collect cause-effect pairs with enough data points from the real world with an authenticated known ground truth. But one must to keep in mind that it is very easy for most of the methods to tune their hyper parameters (even unintentionally) in order to obtain the best results. 

This overfitting problem is often compounded by the fact that this dataset is, most of the time, not separated into train/validation/test sets. To overcome this problem a Cause-effect Pair Challenge has been proposed by Guyon with real and artificial data generated with various mechanisms.

Need for Real Datasets of a Big Size

It is often observed that the cause variable is discrete ordinal while the other is continuous. This induces an artificial asymmetry between cause and effect which could lead to biased assessments.

An open problem could be to find a way to correct this bias when one encounters this type of data. This was done in the design of the dataset of the Cause-effect Pair Challenge.

Biased Assessment Due to Artifacts in Data

The same idea for the cause-effect pair problem for continuous variables could be used for categorical variables or mixed variables. To the best of our knowledge, only few attempts have been made to solve the cause-effect pair problem with categorical data. 

It may be explained by the fact that the cause-effect problem for categorical data may be really harder than for continuous data, because there is in general less information to exploit in the distribution of the noise and in the asymmetry of the causal mechanisms.

Extension of the Generative Approach for Categorical Variables

There is a need for methods that can really cross the bridge between the cause-effect pair problem and the complete problem of causal discovery with more than two variables. A way of research could be to propose an efficient approach for the general multivariate case that can potentially exploit all the information available, including the asymmetry between cause and effect and the conditional independence relations.

Extension of the Pairwise Setting for Complete Graph Inference

Some methods suffer from computational complexity limitations. In the bivariate case, there are only two alternative DAGs to compare, but for the multivariate case with more than 1000 variables, the number of different DAGs to consider grows exponentially. 

Therefore it is a real challenge to make the successful methods of the cause-effect problem scale for big data problem. In particular the methods that can model complex interactions between cause and noise, such as those using Gaussian process regressions or neural networks are often really slow to compute and do not scale well in term of number of variables and number of data points.

Learn Before

Related