To ensure randomness we need to make the following conditions
- minimize allocation bias
- minimize selection bias
- Maximize statistical power :
    - Generally, equal group sizes maximize statistical power, however, unequal groups sizes may be more powerful for some analyses

How do we ensure randomness in a RCT ?

To better understand how we conduct RCT, here is a real example in the book explained in a simpler way.

For example we want to compare two Fertilizer. However, there are many confounders, including weather, soil and etc. Like the Causal Diagram below shows.

So in order to eliminate the confounders, you need to assign the fertilizers randomly into subplots on a land. In this way you can guarantee the majority time you assign fertilizers in a neither most fertile or least fertile subplots. 

Thus, the confounders can be eliminated. Right now, the uncertainty comes from randomlization process rather than the characteristics of the soil. 

Example of RCT

Sensitivity analysis refers to investigating whether the specified outcome could have resulted from alternative hypotheses and how strong those other hypotheses would need to be in order to explain the observed data.

The resulting relationships can be used to contrast the likelihoods with the original hypothesis. 

Sensitivity Analysis

A conventional measure of a treatment's effectiveness. It averages treatment efficacy over a population and can be found using a randomized control trial (RCT). It measures how much an increase in a single unit of treatment variable X will affect the outcome variable Y. When you have properly eliminated all confounding variables, you can obtain an estimate through running a regression.

Average Causal Effect (ACE)

Using RCTs in an online setting. Often deals with testing new features/hypothesis on digital platforms. Companies like microsoft, google and netflix regularly engage in them to improve revenue and UX.

Online Controlled Experiments

Efficacy is the performance of an intervention under ideal and controlled circumstances, whereas effectiveness is how well an intervention works in real-world environments. An intervention can be referred to as efficacious if it demonstrates significant results in controlled experiments, but cannot be referred to as effective until it demonstrates clinical or real-world use. 

Placebo-controlled RCTs are ideal for efficacy evaluation, whereas RCTs comparing an intervention with usual care practices is ideal for effectiveness evaluation.  

Effectiveness- vs. Efficacy-Oriented Randomized Control Trials

Often used in clinical trials, a double-blinded experiment is an experimental technique in which neither the investigators nor the subjects know who is receiving an intervention or not.  It is to prevent bias.  

Double-Blinded Experiment

1. There are experimental units (people, users, etc) that can be parts of different variants (treatment group or control group) that will not interfere (interact, communicate, etc) with one another 

2. There is a large enough sample size of experimental units (people, users, etc)

3. Key metrics (parameters, evaluation criterions, etc) can be calculated and represent the desired measurement

4. The treatment group is practical to make. For online experiments this could mean the software development is practical, and for medical experiments it could be whether it is reasonable to have humans try a potential drug

Necessary ingredients for running useful controlled experiments

A vegetarian boy, Daniel, tried to prove to the overseer of the King that being a vegetarian would not impair his ability to serve the king. For 10 days, the four of them including Daniel were given only vegetables, and the other group was given the king’s meat and wine. After 10 days, the two groups were compared. The result is that Daniel and his three companions grew up healthy on a vegetarian diet. 
- Two groups are comparable and can represent a population. 
- It is important to compare groups instead of individual comparisons.
- Two groups are selected in advance, instead of advertising, only the good ones are shown.

Example of Controlled experience

A case control study is an observational study method that selects people with a disease and those without a disease to investigate their exposure to a risk factor and its degree respectively, so as to determine whether the exposure risk factor is related to a disease and the degree of association.

Drawbacks: (Tobacco case as an example)
-  Retrospective: We know the subject has cancer, and in that context we have to go back and figure out why
-  Reversed logic of probability: What these data tell us is the probability that cancer patients are smokers, not the probability that smokers get cancer. For those who want to know if they should smoke, the probability of cancer in smokers is the probability they really care about
-  Recall bias: While the interviewers may be blinded to cancer vs. control, the patients themselves know whether or not they had cancer. Having already gone through this event can affect their recollections
-  Selection bias: Hospitalized cancer patients (aka those on record as having cancer and those specifically hospitalized for it) are not necessarily representative of the population at large or the smoking population at large.

Advantages of RCTs over Case Control Studies

Advantages of Case Control Studies:
- Quicker, cheaper and require less time and effort than cohort studies
- Case-control studies can study multiple risk factors/exposures
- They are useful for studying outcomes that take a long time to develop

Drawbacks of RCTs: 
- Ethics: RCT approaches are usually controversial and must be handled with care. The experimental group (intervened group) and control group (not- intervened group) are likely to have different outcomes and impact people who get involved in the trial. For example, you cannot assign people to smoke for decades just to figure out if they would get cancer. 
- Feasibility: Running RCTs can be very expensive. They cost time, effort and money. It can take decades to assess the long-term effect of a vaccine. Due to the high costs, most RCTS only inspect one or very few variables. Therefore they rarely reflect the full picture of a complicated issue.


Advantages of Case Control Studies over RCTs


A Randomized Controlled Trial (RCT) or a Controlled Experiment is a method to eliminate unmeasurable confounders effect on causal inference by implementing an intervention on a randomly selected group of individuals (treatment/intervened group) while keeping other individuals unchanged (controlled group, placebo, not-intervened group) to observe the changes in outcomes due to the intervention effect. The RCTs are intended to "disarm" confounders (confounding variables).

University of Michigan - Ann Arbor

"What will happen if we *change* the environment?"
"What if we do..."
Ex: What is the probability we get an A on our next Python exam given we increase our daily study hours?
$ P(AGrade| do(hours))  $


"How?"
Ex: "How many more daily hours should I dedicate to exam study?"

Common Questions of Intervention

Quantitative research is an empirical methodology wherein researchers start with a specific research question, collect numerical data from a sizable sample of participants, and analyze it using statistical techniques. As the most widely used approach in psychology, its primary objective is to deduce generalizable conclusions about a larger population.

Quantitative Research

Causal inference refers to making conclusions about finding the reasonings behind the relationship of two or more variables. After coming to some conclusions about statistical data, we ask why such relationships exist between variables.

Causal Inference

Pearl notes two ways of eliminating unmeasurable confounders, in the context of:

- RCT
- Observational studies

In each of these methods, we can use the front-door criterion, back-door criterion, or if necessary, do-calculus.

Deconfounding/Eliminating Unmeasurable Confounder Variables

Pearl, J., & Mackenzie, D. (2018). $\mathit{The \ Book \ of \ Why: \ The \ New \ Science \ of \ Cause \ and \ Effect.}$ Basic Books.

The Book of Why

Randomized Controlled Trial (RCT) = Controlled Experiment

An observational study, as supposed to an experimental study is a type of study that investigators observe the effect of an intervention without assigning it to the subjects at random.  That is, the subjects choose their own treatment/ intervention.  

Observational studies may prove to be less costly that RCT's, given that the investigator need not actively change the behavior of the subjects


Observational Studies

Qualitative researchers often criticize quantitative methods for overlooking the rich complexity of human behavior and experience. They argue that quantitative approaches tend to oversimplify phenomena by focusing only on answering simple questions about easily quantifiable variables. However, quantitative researchers acknowledge this oversimplification, utilizing it as a deliberate strategy to uncover broad, general principles of human behavior.

Criticisms of Quantitative Research

Group research is the most common quantitative research approach in psychology, characterized by studying a large number of participants and combining their data to draw general conclusions about human behavior. In this methodology, researchers primarily analyze and interpret behavior by examining aggregated statistical metrics across the entire sample, such as group means and standard deviations, rather than focusing on the detailed behavior of individual subjects.

Group Research

The primary strength of quantitative research lies in its ability to provide precise answers to specific, tightly focused research questions and to draw broad, general conclusions about human behavior. By employing statistical analysis on large samples, this methodological approach allows researchers to uncover general behavioral trends, such as the overall tendency of individuals to obey authority figures or comparing traits across demographic groups.

Strengths of Quantitative Research

Single-subject research, frequently referred to as small-$$n$$ design (where $$n$$ represents the statistical symbol for sample size), is a quantitative methodology that involves studying a small number of participants and focusing closely on each individual's behavior. Despite its name, this approach typically investigates between two and ten participants rather than just one. Serving as a crucial alternative to group research, particularly in applied areas of psychology, single-subject research seeks to comprehend objective behavior by utilizing strict experimental manipulation and control to collect highly structured data that is analyzed using quantitative techniques.

Single-Subject Research

Despite its ability to yield precise answers and general conclusions, quantitative research is inherently limited in its capacity to generate novel or unexplored research questions. Furthermore, it is not well-suited for providing rich, detailed descriptions of specific groups' behaviors within particular contexts, nor does it effectively convey the subjective, lived experience of being a member of a given group.

Weaknesses of Quantitative Research

Qualitative and quantitative research differ significantly in their methodology and goals. Quantitative research generally starts with a focused hypothesis, gathering numerical data from large samples to draw general population conclusions using statistical techniques. In contrast, qualitative research begins with a broader, less focused research question and collects large amounts of unfiltered, nonnumerical data from a small number of individuals. Qualitative analysis relies on nonstatistical techniques to deeply understand the detailed experiences of participants rather than generating broad statistical generalizations.

Comparison of Qualitative and Quantitative Research

Survey research is a flexible methodology that incorporates both quantitative and qualitative approaches, defined by two primary characteristics. First, it relies on self-report measures, where researchers ask participants to directly report on their thoughts, feelings, and behaviors using tools like questionnaires or interviews. Second, it emphasizes rigorous sampling techniques, typically favoring large random samples to acquire accurate estimates of population characteristics. This makes survey research one of the few approaches in psychology that routinely relies on random sampling.

Characteristics of Survey Research

Which of the following statements best describes the methodology and primary objective of quantitative research in psychology?

A machine passes this test if a human could perceive their interactions with the machine as human, indicating that the machine has successfully emulated human reasoning. 

Turing Test

- Causal Inference Books
- Causal Inference Papers

Causal Inference References

The calculus of causation is formed of:
1. Causal diagrams (the language of knowledge)
2. Causal symbolic language (the language of queries)

The calculus of causation

Organisms, as well as ML, can be divided into 3 levels of causation corresponding to three cognitive abilities: seeing, doing, and imagining.
1. Association
2. Intervention
3. Counterfactuals


Ladder of Causation 

Bayes Theorem is about predicting the probability of an event A occurring given an event B has already occurred. We can use the inverse probability of this statment and the probability of each event to find this value. 
P(A) is called prior probability, P(B|A) is Likelihood, P(B) is evidence, and P(A|B) is posterior probability.


Bayes Theorem Overview

Most of the statistical methods are entirely objective, which means reasoning exclusively from data and experiments. But causal analysis and Bayesian analysis are different in that they involves some subjectivity.

Causal analysis requires some subjective commitments based on one's knowledge in the field before the causal digram is drawn and data is explored.

Similarly, Bayesian analysis combines objective evidence with the prior subjective belief.

From objectivity to subjectivity

Both Induction and Deduction are necessary within casual inference because they are two ways of approaching a problem and often go hand in hand. Induction goes from evidence -> hypothesis while deduction works in the forward direction; cause (hypothesis) -> effect (theory, conclusion). Artificial Intelligence has started automating Induction and Deduction, an example is Bayesian networks. 

Stages of Casual Inference: Induction and Deduction

Logical reasoning is the process of using a rational, systematic series of steps based on sound certain mathematical procedures and statements to arrive at a conclusion. 

There are three types of the reasoning:
- Abductive Reasoning 
- Deductive Reasoning 
- Inductive Reasoning

Reasoning

Hill's Criteria is a collection of nine informal guidelines ("viewpoints" as supposed to requirements) used to determine whether a causal relationship exists within an observed association.  Initially, five out of the nine viewpoints were proposed by the surgeon general's advisory committee, to which Bradford Hill later added four more.  Hill's Criteria is not meant to be the necessary conditions to support or refute a cause-and-effect hypothesis.  The debate over the causality between cigarette and lung cancer prompted the criteria, which were used for the proponent of the causal relationship to justify their claim that cigarette smoking is indeed causally related to lung cancer.  

Hill's Criteria

1. necessary causation (but-for causation)
2. sufficient causation
3. necessary-and-sufficient causation

Three different kinds of causation

1. The Law of Counterfactuals
2. The Law of Conditional Independence

The Two Fundamental Laws of Causal Inference

List of techniques that help solve intractable inference problems in machine learning that usually arise from interactions between latent variables in a structured graphical model.

-Expectation Maximization

-MAP Inference and Sparse Coding

-Variational Inference and Learning

-Learned Approximate Inference



Approximate Inference 

"The estimand is the target quantity—the purpose of the statistical analysis ... the thing we are estimating" " that connects statistical evidence to theory."

It refers "to both the theoretical and the empirical estimands."

Estimand

1- "Set a **theoretical estimand**, clearly connecting this quantity to theory"

2- "Link to an **empirical estimand**, which is informative about the theoretical estimand under some identification assumptions"

3- Use "an **estimation strategy** to learn the empirical estimand (e.g., a regression model) ... from data."

Three Critical Choices in Causal Inference

- Correlation does not imply causation.
- Causation does not imply correlation either.

Correlation vs. Causation

Establishing causality in economics is a significant challenge for two main reasons. First, unlike in many natural sciences, it is often impossible to conduct controlled experiments to isolate a single variable. The economy is a dynamic system where numerous factors change simultaneously, creating confounding variables that can lead to misleading correlations. Second, the economy's fundamental complexity, arising from the interactions of millions of people, makes it impossible to measure and understand every variable. This inherent complexity means economists must be skeptical of simple claims that one complex phenomenon, like capitalism, is the direct cause of another, such as a rise in living standards. To address these challenges, economists use methods like natural experiments and advanced statistical techniques to infer causal relationships from observational data.

The Challenge of Establishing Causality in Economics

For Causal Nodes X,Y:
X and Y will be deconfounded if we block every backdoor, because such paths allow for spurious correlations. 

When eliminating backdoors, we additionally need to ensure that anything closed off from X is not a descendent of X.

Back-Door Criterion

Pearl introduces an axiomatic system containing three rules to remove do operators in Chapter 7. 

Rule 1. Irrelevance
Rule 2. Conditional equivalence of *do(x)* and *see(x)*
Rule 3. Lack of causal paths

Generalizing Do-Calculus

When the causal effect of a variable $X$ on $Y$ is mediated by a variable $M$, which is shielded from a confounding set $C$, if $C$ is *unmeasurable*, we cannot block the back-door $X \leftarrow C \rightarrow Y$. So, we use the front-door $X \rightarrow M \rightarrow Y$. 

Front-Door Criterion (Unmeasurable confounder of a mediated path)

$\ \ \ \ \ \ \ \ \ \ \ C$

$\ \ \ \ \swarrow \ \ \ \ \ \ \ \ \searrow$

$X \rightarrow M \rightarrow Y$

Learn Before

Related

Learn After