A conventional measure of a treatment's effectiveness. It averages treatment efficacy over a population and can be found using a randomized control trial (RCT). It measures how much an increase in a single unit of treatment variable X will affect the outcome variable Y. When you have properly eliminated all confounding variables, you can obtain an estimate through running a regression.

University of Michigan - Ann Arbor

A unidirectional graph of nodes and arrows, where each node represents a variable (measure) under the study and the arrows represent how the nodes causally affect each other. "The Book of Why" calls them the language of knowledge.

Instead of causal diagrams, econometricians use mathematical equations and statisticians use a set of assumptions that characterize the arrows in causal diagrams. 

A causal diagram, regardless of its method of demonstration, should illustrate the cause-effect relations in the environment that have affected the data generation process.

Causal diagram (for Path analysis)

The regression coefficient is $$\beta_1$$, in the regression equation $$y \approx \beta_0 + \beta_1 X$$, is an esitmate of the rate of change (slope) between X and Y.

Regression Coefficient

Pearl introduces an axiomatic system containing three rules to remove do operators in Chapter 7. 

Rule 1. Irrelevance
Rule 2. Conditional equivalence of *do(x)* and *see(x)*
Rule 3. Lack of causal paths

Generalizing Do-Calculus

A Randomized Controlled Trial (RCT) or a Controlled Experiment is a method to eliminate unmeasurable confounders effect on causal inference by implementing an intervention on a randomly selected group of individuals (treatment/intervened group) while keeping other individuals unchanged (controlled group, placebo, not-intervened group) to observe the changes in outcomes due to the intervention effect. The RCTs are intended to "disarm" confounders (confounding variables).

Randomized Controlled Trial (RCT) = Controlled Experiment

Pearl, J., & Mackenzie, D. (2018). $\mathit{The \ Book \ of \ Why: \ The \ New \ Science \ of \ Cause \ and \ Effect.}$ Basic Books.

The Book of Why

As defined in "The Book of Why", causal diagrams are Bayesian networks where every arrow indicates either a direct causal relation or the possibility of a direct causal relation in the direction that the arrow is facing. Unlike causal diagrams, Bayesian networks do not have to be based on causal relations.

The meaning of the arrows is a major difference between Bayesian networks and causal diagrams. 

In a probabilistic Bayesian network, the arrows into $Y$ mean that the probability of Y is governed by the conditional probability tables for $Y$, given observations of its parent variables. The same is true for causal Bayesian networks, except that the conditional probability tables specify the probability of $Y$ given interventions on the parent variables. Both models specify probabilities for $Y$, not a specific value of $Y$. 

In a structural causal model, there are no conditional probability tables. The arrows simply mean $Y$ is a function of its parents, as well as the exogenous variable $U_Y$:
$Y = f_Y(X, A, B, C,…, U_Y)$


Relation between Bayesian Networks and Causal Diagrams

Simplified causal diagram describing a firing squad 

Causal Diagram Example

Pearl outlines 3 basic types of junctions in Causal Diagrams which appear commonly and help explain the data. 

The three junctions take the forms:
   1. $A \rightarrow B \rightarrow C$ (Chain)
 2. $A \leftarrow B \rightarrow C$ (Fork)
 3. $A \rightarrow B \leftarrow C$ (Collider)

The 3 Junctions in Causal Diagram

Staplin, N., Herrington, W. G., Judge, P. K., Reith, C. A., Haynes, R., Landray, M. J., ... & Emberson, J. (2017). Use of causal diagrams to inform the design and interpretation of observational studies: an example from the Study of Heart and Renal Protection (SHARP). Clinical Journal of the American Society of Nephrology, 12(3), 546-552.

Use of Causal Diagrams to Inform the Design and Interpretation of Observational Studies: An Example from the Study of Heart and Renal Protection (SHARP)

- Ancestors
- Descendants
- Children
- Parents

Types of relations in a Causal Diagram

Average Causal Effect (ACE)

Path analysis is the use of linear regression and multiple regression techniques to evaluate relationships between multiple dependent variables and multiple independent variables. Lines are drawn unidirectionally based on correlation/association between two variables. Path analysis is often thought of as the earliest form of statistical analysis closest to causal inference despite still being association-based and can be thought of as a predecessor to Bayesian networks.

Path Analysis 

Streiner, D. L. (2005). Finding Our Way: An Introduction to Path Analysis. The Canadian Journal of Psychiatry, 50(2), 115-122. doi:10.1177/070674370505000207
Link: https://journals.sagepub.com/doi/pdf/10.1177/070674370505000207

Introduction to Path Analysis 

"Each theoretical estimand is linked to an empirical estimand involving only observable quantities (e.g., a difference in means in a population) by assumptions about the relationship between the data we observe and the data we do not. These identification assumptions can be conveyed in a Directed
Acyclic Graph (DAG. i.e., a causal diagram)."

Empirical Estimand

For the equation $\hat{y}=\bar{\beta}_0+\bar{\beta}_1x$, RSS is minimized (and the regression is "best") when 
$\hat{\beta}_1=\dfrac{\sum_{i=1}^n (x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^n (x_i-\bar{x})^2}$ 
and 
$\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}$, 
where $\bar{x},\bar{y}$ are the sample means. 

Least Squares Coefficient Estimates

When there is a condounder $Z$ for $X$ and $Y$, by plotting all three variables together, with each value of $(X, Y, Z)$ describing one point in space. In this case, the data will form a cloud of points in XYZ-space. The analogue of a regression line is a regression plane, which has an equation that looks like $Y = aX + bZ + c$. We can easily compute $a, b, c$ from the data.

The coefficient $a$ gives us the regression coefficient of $Y$ on $X$ already adjusted for $Z$. (It is called a partial regression coefficient and written $r_{YX.Z}$.)

If $Z$ is the only confounder, then a is the average causal effect of $X$ on $Y$.

Partial Regression Coefficient

When we observe a variable W that is irrelevant to Y (possibly conditional on other variables Z), then the probability distribution of Y will not change. 

This assertion of irrelevance translates into a symbolic manipulation:
$P(Y | do(X), Z, W) = P(Y | do(X), Z)$

The stated equation holds provided that the variable set $Z$ blocks all the paths from $W$ to $Y$ after we have deleted all the arrows leading into $X$.


Rule 1. Irrelevance

If a set Z of variables blocks all back-door paths from $X$ to $Y$, then conditional on $Z$, $do(X)$ is equivalent to $see(X)$. We can, therefore, write
$P(Y | do(X), Z) = P(Y | X, Z)$
if $Z$ satisfies the back-door criterion.

Rule 2: Conditional Equivalence of $do(x)$ and $see(x)$

We can remove $do(X)$ from $P(Y| do(X))$ in any case where there are no causal paths from $X$ to $Y$. That is, 
$P(Y | do(X)) = P(Y)$
if there is no path from $X$ to $Y$ with only forward-directed arrows.

We can paraphrase this rule is follows: if we do something that does not affect $Y$, then the probability distribution of $Y$ will not change.

Rule 3. Lack of Causal Paths

Derivation of the Front-Door Adjustment Formula from the Rules of Do-Calculus

Rule 1 permits the addition or deletion of observations. 
Rule 2 permits the replacement of an intervention with an observation, or vice versa. 
Rule 3 permits the deletion or addition of interventions. 

All of these permits are issued under appropriate conditions, which have to be verified in any particular case from the causal diagram.


A Simple Syntactic Interpretation for the Rules

To ensure randomness we need to make the following conditions
- minimize allocation bias
- minimize selection bias
- Maximize statistical power :
    - Generally, equal group sizes maximize statistical power, however, unequal groups sizes may be more powerful for some analyses

How do we ensure randomness in a RCT ?

To better understand how we conduct RCT, here is a real example in the book explained in a simpler way.

For example we want to compare two Fertilizer. However, there are many confounders, including weather, soil and etc. Like the Causal Diagram below shows.

So in order to eliminate the confounders, you need to assign the fertilizers randomly into subplots on a land. In this way you can guarantee the majority time you assign fertilizers in a neither most fertile or least fertile subplots. 

Thus, the confounders can be eliminated. Right now, the uncertainty comes from randomlization process rather than the characteristics of the soil. 

Example of RCT

Sensitivity analysis refers to investigating whether the specified outcome could have resulted from alternative hypotheses and how strong those other hypotheses would need to be in order to explain the observed data.

The resulting relationships can be used to contrast the likelihoods with the original hypothesis. 

Sensitivity Analysis

Using RCTs in an online setting. Often deals with testing new features/hypothesis on digital platforms. Companies like microsoft, google and netflix regularly engage in them to improve revenue and UX.

Online Controlled Experiments

Efficacy is the performance of an intervention under ideal and controlled circumstances, whereas effectiveness is how well an intervention works in real-world environments. An intervention can be referred to as efficacious if it demonstrates significant results in controlled experiments, but cannot be referred to as effective until it demonstrates clinical or real-world use. 

Placebo-controlled RCTs are ideal for efficacy evaluation, whereas RCTs comparing an intervention with usual care practices is ideal for effectiveness evaluation.  

Effectiveness- vs. Efficacy-Oriented Randomized Control Trials

Often used in clinical trials, a double-blinded experiment is an experimental technique in which neither the investigators nor the subjects know who is receiving an intervention or not.  It is to prevent bias.  

Double-Blinded Experiment

1. There are experimental units (people, users, etc) that can be parts of different variants (treatment group or control group) that will not interfere (interact, communicate, etc) with one another 

2. There is a large enough sample size of experimental units (people, users, etc)

3. Key metrics (parameters, evaluation criterions, etc) can be calculated and represent the desired measurement

4. The treatment group is practical to make. For online experiments this could mean the software development is practical, and for medical experiments it could be whether it is reasonable to have humans try a potential drug

Necessary ingredients for running useful controlled experiments

A vegetarian boy, Daniel, tried to prove to the overseer of the King that being a vegetarian would not impair his ability to serve the king. For 10 days, the four of them including Daniel were given only vegetables, and the other group was given the king’s meat and wine. After 10 days, the two groups were compared. The result is that Daniel and his three companions grew up healthy on a vegetarian diet. 
- Two groups are comparable and can represent a population. 
- It is important to compare groups instead of individual comparisons.
- Two groups are selected in advance, instead of advertising, only the good ones are shown.

Example of Controlled experience

A case control study is an observational study method that selects people with a disease and those without a disease to investigate their exposure to a risk factor and its degree respectively, so as to determine whether the exposure risk factor is related to a disease and the degree of association.

Drawbacks: (Tobacco case as an example)
-  Retrospective: We know the subject has cancer, and in that context we have to go back and figure out why
-  Reversed logic of probability: What these data tell us is the probability that cancer patients are smokers, not the probability that smokers get cancer. For those who want to know if they should smoke, the probability of cancer in smokers is the probability they really care about
-  Recall bias: While the interviewers may be blinded to cancer vs. control, the patients themselves know whether or not they had cancer. Having already gone through this event can affect their recollections
-  Selection bias: Hospitalized cancer patients (aka those on record as having cancer and those specifically hospitalized for it) are not necessarily representative of the population at large or the smoking population at large.

Advantages of RCTs over Case Control Studies

Advantages of Case Control Studies:
- Quicker, cheaper and require less time and effort than cohort studies
- Case-control studies can study multiple risk factors/exposures
- They are useful for studying outcomes that take a long time to develop

Drawbacks of RCTs: 
- Ethics: RCT approaches are usually controversial and must be handled with care. The experimental group (intervened group) and control group (not- intervened group) are likely to have different outcomes and impact people who get involved in the trial. For example, you cannot assign people to smoke for decades just to figure out if they would get cancer. 
- Feasibility: Running RCTs can be very expensive. They cost time, effort and money. It can take decades to assess the long-term effect of a vaccine. Due to the high costs, most RCTS only inspect one or very few variables. Therefore they rarely reflect the full picture of a complicated issue.


Learn Before

Related