Pharmaceutical companies are interested in determining the influence of genes on each other. A dataset may consist of jointly recorded gene activities for $N$ pairs of genes $( X_k , Y_k ), k = 1, ⋯ N$ and for $n$ patients $S_k = \{( x_{k1} , y_{k 1 }), ⋯ , ( x_{ kn} , y_{kn} \} $. 

Determining which gene influences which other gene is a costly experimental process, so only a subset of pairs of genes are generally labeled with ground truth of causal relationship $G = g_k$ . 

The labeled dataset $\{( S_1 , g_1 ), ⋯ , ( S_k , g_k ), ⋯\}, k = 1⋯ N$ , is an empirical sample of the “mother distribution” over pairs of genes and their causal labels, which may be used as training data to obtain a classifier, such that predictions of unknown causal labels can be made on new pairs of genes.

Example 1: Genomics

A country conducts a census survey, including socio-demographic questions (age, gender, education, profession, number of children, salary, etc.). The question is to determine which factor influences which other factor. 

The dataset consists of joint answers from $n$ citizens for factor pairs $( X_k , Y_k ), k = 1, ⋯ N : S_k = \{( x_{k 1} , y_{k 1} ), ⋯ , ( x_{kn} , y_{kn} \} $. Prior knowledge allows us to determine the ground truth for a few pairs $( X_k , Y_k )$ (for example age can influence wages and not vice-versa). 

This constitutes an empirical “mother distribution” of pairs $\{( S_1 , g_1 ), ⋯ , ( S_k , g_k ), ⋯ , ( S_N , g_N )\} $ from which we might train a classifier to label other similar pairs with their causal direction.

Example 3: Social Science

To illustrate a mother distribution in causal discovery, consider an ecological study conducted to understand a forest ecosystem and preserve biodiversity. The objective is to determine which factor influences which other factor. The dataset consists of $$n$$ jointly recorded values of many variables $$(X_k, Y_k)$$, where $$k = 1, \dots, N$$, in different locations (measuring factors such as soil, humidity, lighting, or the presence of certain plants or animals). This results in samples $$S_k = \{ (x_{k1}, y_{k1}), \dots, (x_{kn}, y_{kn}) \}$$. Determining which factor influences another is a complicated process; however, prior knowledge of physics and biology may allow researchers to label some pairs with ground truth $$G = g_k$$ (for example, the aspect of a slope can influence hill shade, but not vice versa). The labeled dataset thus obtained, $$\{ (S_1, g_1), \dots, (S_N, g_N) \}$$, is an empirical sample of the "mother distribution". It is hoped that more pairs can be automatically labeled using a classifier trained on such data, provided the mechanisms of the other pairs bear some similarity.

Example of a Mother Distribution in Ecology

The meaning of a "mother distribution" can be clarified through examples of causal discovery problems in genomics, ecology, and social sciences.

For notational simplicity, we sometimes reformulate the distribution $$P_{Π = π} ( X , Y )$$ as $$P ( X_π , Y_π )$$, representing the distribution over two random variables indexed by $$π$$. In these examples, $$π$$ is simply a discrete index $$k$$.

University of Michigan - Ann Arbor

Google

The mother distribution is a distribution over joint distributions of pairs of variables. The authors adopt the following definition of a mother distribution: A mother distribution $$P_M( P_{Π = π} ( X , Y ), G = g )$$ is a distribution over pairs $$( P_{Π = π} ( X , Y ), G = g )$$, where $$Π$$ and $$G$$ are random variables: - $$Π = π$$ indexes distribution $$P_{Π = π} ( X , Y )$$ over random variables $$X$$ and $$Y$$; $$π$$ may possibly take an infinite number of values (e.g. parameters of a generative process) - $$G = g$$ labels causal relationship, $$g ∈ \{ X → Y , X ← Y , X \leftrightarrow Y , X ⊥ Y \}$$.

Mother Distribution

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

Practically, a specific distribution $$P_{Π = π} ( X , Y )$$ is only known through a finite sample $$S_{Π = π} ( X , Y ) = \{( x_1 , y_1 ), ( x_2 , y_2 ), \cdots , ( x_n , y_n )\}$$ (a scatter plot). Hence, we can define a double random process for dataset generation: 1. Draw a pair $$(P_{Π = π} ( X , Y ), G=g)$$ from the mother distribution $$P_M(P_{Π = π} ( X , Y ),G)$$, where $$G=g$$ labels the causal relationship. 2. Draw $$n$$ samples from $$P_{Π = π} ( X , Y )$$ to form $$S_{Π = π} ( X , Y )$$. 3. Repeat the process to obtain enough training and/or test examples: $$\{ (S_1(X,Y),g_1), (S_2(X,Y),g_2), \cdots, (S_N(X,Y),g_N) \}$$.

Learn Before

Related

Learn After