University of Michigan - Ann Arbor

Cinelli, Carlos, and Judea Pearl. “Generalizing Experimental Results by Leveraging Knowledge of Mechanisms.” Technical Report - UCLA, 10 Dec. 2019.

https://ftp.cs.ucla.edu/pub/stat_ser/r492.pdf

Generalizing Experimental Results by
Leveraging Knowledge of Mechanisms

The results of RCTs must be generalized when the mechanisms that generate the output differ between the population and the sample. Generalization is done by recognizing what variables control the output, and measuring them. If measuring the variables directly responsible is not possible, we 'isolate' them by measuring other appropriate variables.

Generalization in Randomized Control Trials

Key Idea

The city of Los Angeles runs an RCT to determine effect of Russian Roulette on mortality. Death rate amongst Roulette players is 17.5%. Death rate among others is 1%.
The mayor of New York now wants to determine what the city's mortality rate would be if everyone plays Russian Roulette. New York starts with a mortality rate of 5% (vs LA's 1%).
How do we use the data from the LA RCT to determine this answer?

Motivating Example : Russian Roulette

Some useful definitions to better understand the paper.
1. Selection Nodes
2. Functional Constraints
3. Transportability
4. Monotonicity

Helpful Definitions for the Paper

Preferable when target quantity isn't observable from the data i.e. observed variables have some relationship to target variables, but the target isn't observed.

Bayesian Approach

1. Functional constraints are important when trying to generalize RCTs
2. Experimental data from multiple sources allows for point estimation

Conclusions of Paper

When using Structural Causal Models (SCMs) to generalize experimental results, two key assumptions are typically made:

1. The causal factors that affect the outcome are independent.
2. There are functional constraints on these causal factors.

Learn Before

Related