Pharmaceutical companies are interested in determining the influence of genes on each other. A dataset may consist of jointly recorded gene activities for $N$ pairs of genes $( X_k , Y_k ), k = 1, ⋯ N$ and for $n$ patients $S_k = \{( x_{k1} , y_{k 1 }), ⋯ , ( x_{ kn} , y_{kn} \} $. 

Determining which gene influences which other gene is a costly experimental process, so only a subset of pairs of genes are generally labeled with ground truth of causal relationship $G = g_k$ . 

The labeled dataset $\{( S_1 , g_1 ), ⋯ , ( S_k , g_k ), ⋯\}, k = 1⋯ N$ , is an empirical sample of the “mother distribution” over pairs of genes and their causal labels, which may be used as training data to obtain a classifier, such that predictions of unknown causal labels can be made on new pairs of genes.

Example 1: Genomics

A study is conducted to understand better an ecosystem in a forest area and preserve biodiversity. The question is to determine which factor influences which other factor. 

The dataset consists of $n$ jointly recorded values of many variables $( X_k , Y_k ), k = 1, ⋯ N$ , in different locations, such as soil, humidity, lighting, presence of certain plants or animals, etc., resulting in samples $S_k = \{( x_{k 1} , y_{k 1} ), ⋯ , ( x_{kn} , y_{kn} \} $. 

Determining which factor influences which other factor is a complicated process, however, prior knowledge of physics and biology may allow us to label some pairs with ground truth $G = g_k$ (for example the aspect of a slope can influence hill shade and not vice-versa). The labeled dataset thus obtained $\{( S_1 , g_1 ), ⋯ , ( S_k , g_k ), ⋯\}, k = 1⋯ N$ , is an empirical sample of the “mother distribution”. 

It is hoped that we can label automatically more pairs with a classifier trained on such data, if the mechanisms of the other pairs bear some similarity.

Example 2: Ecology

A country conducts a census survey, including socio-demographic questions (age, gender, education, profession, number of children, salary, etc.). The question is to determine which factor influences which other factor. 

The dataset consists of joint answers from $n$ citizens for factor pairs $( X_k , Y_k ), k = 1, ⋯ N : S_k = \{( x_{k 1} , y_{k 1} ), ⋯ , ( x_{kn} , y_{kn} \} $. Prior knowledge allows us to determine the ground truth for a few pairs $( X_k , Y_k )$ (for example age can influence wages and not vice-versa). 

This constitutes an empirical “mother distribution” of pairs $\{( S_1 , g_1 ), ⋯ , ( S_k , g_k ), ⋯ , ( S_N , g_N )\} $ from which we might train a classifier to label other similar pairs with their causal direction.

Example 3: Social Science

The meaning of “mother distribution” may seem mysterious. Here are few examples of causal discovery problems, which could clarify this notion. 
 - Example 1: Genomics
 - Example 2: Ecology
 - Example 3: Social Sciences

For notational simplicity we sometimes reformulate $P_{Π = π} ( X , Y )$ as $P ( X_π , Y_π )$, the distribution over two random variables indexed by $π$ . In the examples, $π$ is just a discrete index $k$ .

University of Michigan - Ann Arbor

The mother distribution is a distribution over join distributions of pairs of variables. The authors adopt the following definition of a mother distribution:

A mother distribution $P_M( P_{Π = π} ( X , Y ), G = g )$ is a distribution over pairs $( P_{Π = π} ( X , Y ), G = g )$, where $Π$ and $G$ are random variables:
 - $Π = π$ indexes distribution $P_{Π = π} ( X , Y )$ over random variables $X$ and $Y$ ; $π$ may possibly take an infinite number of values (e.g. parameters of a generative process)
 - $G = g$ labels causal relationship, $g ∈ \{ X → Y , X ← Y , X \leftrightarrow Y , X ⊥ Y \}$.

Mother Distribution

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

Examples of Mother Distribution

Practically, $P_{Π = π} ( X , Y )$ is only known through a finite sample $S_{Π = π} ( X , Y ) = \{( x_1 , y_1 ), ( x_2 , y_2 ), ⋯ , ( x_n , y_n )\} $ (a scatter plot ). Hence, we have a double random process:
1. Draw Pair $(P_{Π = π} ( X , Y ), G=g)$ from mother distribution $P_M(P_{Π = π} ( X , Y ),G)$.
2. Draw $n$ samples from $P_{Π = π} ( X , Y )$:
$S_{Π = π} ( X , Y ) = \{( x_1 , y_1 ), ( x_2 , y_2 ), ⋯ , ( x_n , y_n )\} $
3. Repeat the process to obtain emough training and/or test examples:
$\{ (S_1(X,Y),g_1),  (S_2(X,Y),g_2), \cdots, (S_N(X,Y),g_N)  \} $

Learn Before

Related

Learn After