Learn Before
Concept icon
Concept

Kolmogorov Complexity in Causal Inference

In causal inference, the basic postulate that "the factorization of the joint density function Pcause,effectP_{cause,effect} into PcausePeffect∣causeP_{cause} P_{effect|cause} should lead to a simpler model than PeffectPcause∣effectP_{effect} P_{cause|effect}" can be expressed using the Kolmogorov complexity framework: K(Pcause)+K(Peffect∣cause)≤+K(Peffect)+K(Pcause∣effect)K(P_{cause}) + K(P_{effect|cause}) \overset{+}\leq K(P_{effect}) + K(P_{cause|effect}) This inequality derives from the postulate of algorithmic independence between the distribution of the cause PcauseP_{cause} and the distribution of the causal mechanism Peffect∣causeP_{effect|cause}, stated by Janzing and Schölkopf as: I(Pcause:Peffect∣cause)=+0I(P_{cause} : P_{effect|cause}) \overset{+}=0 where I(Pcause:Peffect∣cause)I(P_{cause} : P_{effect|cause}) denotes algorithmic mutual information. Since Kolmogorov complexity and algorithmic mutual information are not computable in practice, they have inspired practical implementations such as model selection with the Minimum Message Length (MML) principle and methods exploiting the independence between cause and mechanism.

0

1

Concept icon
Updated 2026-06-16

Tags

Data Science