When two researchers have different casual diagrams they can analyze the same dataset and can come up with different conclusions due to the subjectivity in their diagrams. 

Although when two researchers agree on the same assumptions it allows a very objective way of interperting data as well as new data.   

Interpeting Data with different Casual Diagrams

Most of the statistical methods are entirely objective, which means reasoning exclusively from data and experiments. But causal analysis and Bayesian analysis are different in that they involves some subjectivity.

Causal analysis requires some subjective commitments based on one's knowledge in the field before the causal digram is drawn and data is explored.

Similarly, Bayesian analysis combines objective evidence with the prior subjective belief.

University of Michigan - Ann Arbor

Bayesian networks were developed to understand reasoning under conditions of uncertainty. It is based on Bayes Theorem and is a probabilistic model that represents a set of variables and their conditional dependencies by being able to find the probability of facts being true/false given already defined facts. A Bayesian network is represented by a directed acyclic graph which helps to visualize the probabilistic model for reviewing relationships between random variables and reasonings for causal probabilities given certain facts



Bayesian Networks

Causal inference refers to making conclusions about finding the reasonings behind the relationship of two or more variables. After coming to some conclusions about statistical data, we ask why such relationships exist between variables.

Causal Inference

Pearl, J., & Mackenzie, D. (2018). $\mathit{The \ Book \ of \ Why: \ The \ New \ Science \ of \ Cause \ and \ Effect.}$ Basic Books.

The Book of Why

A Bayesian network's structure is made up of nodes (which can be multi-variate), edges between paired nodes, and a conditional probability distribution in each nodes. The nodes in a Bayesian network represent random variables and edges represent the conditional dependence of these nodes. We try to use this structure to model the conditional probability distribution of outcome variables (which are often causal) after observing new evidence.

How do Bayesian Networks work?

This article covers Bayesian networks and how they work by looking at different aspects of probabilistic graphical models.

[www.bayesserver.com/docs/introduction/bayesian-networks#_dynamic-bayesian-networks](https://www.bayesserver.com/docs/introduction/bayesian-networks#_dynamic-bayesian-networks) 

Helpful article about Bayesian Networks

Belief Propagation is an iterative message passing algorithm used within graphs or trees. In Bayesian Networks, bayesian likelihoods are calculated at certain nodes and passed up or down the network.

* If the message is passed downward (e.g. from parent to child), the child can update its beliefs using conditional probabilities.
* If the message is passed upward (e.g. from child to parent), the parent can update its beliefs by multiplying them by a likelihood ratio.

Belief Propagation in Bayesian Networks

Looking at related information based on the value of one variable is called conditioning on a variable. This is a helpful tool when programming a machine to update its beliefs: it can focus on relevant information while ignoring the rest.

Conditioning on a Variable

The conditional probability table displays conditional probabilities of mutually dependent random variables of a single variable contingent on the others.  Not to be confused with the Contingency Table, which describes the frequencies of data points in the given category. 

Conditional Probability Table

As mentioned before, Bayesian networks encode conditional independencies, i.e. we can use them to determine "independent" events.

Bayesian Networks and Independence

"Bayesian intelligence failed to bridge the gap between artificial and human intelligence. I'm sure you can figure out the missing ingredient: causality"



Criticism of Bayesian Network from the Judea Pearl

From objectivity to subjectivity

Given that the guest chooses door 1 and the host opens door 3, we have: 


$P(1=car|3=opened) $

$= \frac{P(3=opened|1=car) *P(1=car)} {p(3=opened)} $

$=\frac{\frac{1}{2}*\frac{1}{3}}{p(3=opened)}$


P(3=opened)  is the probability that the host opens door 3 when guest picked door 1.

$P(3=opened)$ 
$= P(3=opened |1= car)*P(1= car)+P(3=opened|2= car)*P(2 = car)+P(3=opened |3 = car)*P(3 = car)$
$=\frac{1}{2}*\frac{1}{3}+1*\frac{1}{3}+0*\frac{1}{3}$
$=\frac{1}{2}$


$P(1=car|3=opened) $
$=\frac{\frac{1}{6}}{\frac{1}{2}}$
$=\frac{1}{3}$

Hence the possibility of car is behind door 1 is $1/3$, given guest selected door 1 and host opened door 3. 

Bayesian Solution for Monty Hall’s Paradox


- The COVID-19 testings tend to focus on already hospitalized people, not accounting for the people with mild symptoms and the asymptomatics leading to selection bias.  
- The differences in death rates across countries may not be attributed to clinical, demographic, and environmental factors.  
- Together with random sampling/testing, the Bayesian Network model allows causal explanations from data for better predictions.  


Summary of "COVID-19 infection and death rates: the need to incorporate causal explanations for the data and avoid bias in testing"

Structure Learning is a field of machine learning in which data is used to learn the links of a Bayesian network or Dynamic Bayesian network. The two major approaches for structure learning are the score-based approach and the constraint-based approach.

Structure Learning

A machine passes this test if a human could perceive their interactions with the machine as human, indicating that the machine has successfully emulated human reasoning. 

Turing Test

- Causal Inference Books
- Causal Inference Papers

Causal Inference References

The calculus of causation is formed of:
1. Causal diagrams (the language of knowledge)
2. Causal symbolic language (the language of queries)

The calculus of causation

Organisms, as well as ML, can be divided into 3 levels of causation corresponding to three cognitive abilities: seeing, doing, and imagining.
1. Association
2. Intervention
3. Counterfactuals


Ladder of Causation 

Bayes Theorem is about predicting the probability of an event A occurring given an event B has already occurred. We can use the inverse probability of this statment and the probability of each event to find this value. 
P(A) is called prior probability, P(B|A) is Likelihood, P(B) is evidence, and P(A|B) is posterior probability.


Bayes Theorem Overview

Both Induction and Deduction are necessary within casual inference because they are two ways of approaching a problem and often go hand in hand. Induction goes from evidence -> hypothesis while deduction works in the forward direction; cause (hypothesis) -> effect (theory, conclusion). Artificial Intelligence has started automating Induction and Deduction, an example is Bayesian networks. 

Stages of Casual Inference: Induction and Deduction

Logical reasoning is the process of using a rational, systematic series of steps based on sound certain mathematical procedures and statements to arrive at a conclusion. 

There are three types of the reasoning:
- Abductive Reasoning 
- Deductive Reasoning 
- Inductive Reasoning

Reasoning

Hill's Criteria is a collection of nine informal guidelines ("viewpoints" as supposed to requirements) used to determine whether a causal relationship exists within an observed association.  Initially, five out of the nine viewpoints were proposed by the surgeon general's advisory committee, to which Bradford Hill later added four more.  Hill's Criteria is not meant to be the necessary conditions to support or refute a cause-and-effect hypothesis.  The debate over the causality between cigarette and lung cancer prompted the criteria, which were used for the proponent of the causal relationship to justify their claim that cigarette smoking is indeed causally related to lung cancer.  

Hill's Criteria

1. necessary causation (but-for causation)
2. sufficient causation
3. necessary-and-sufficient causation

Three different kinds of causation

1. The Law of Counterfactuals
2. The Law of Conditional Independence

The Two Fundamental Laws of Causal Inference

A Randomized Controlled Trial (RCT) or a Controlled Experiment is a method to eliminate unmeasurable confounders effect on causal inference by implementing an intervention on a randomly selected group of individuals (treatment/intervened group) while keeping other individuals unchanged (controlled group, placebo, not-intervened group) to observe the changes in outcomes due to the intervention effect. The RCTs are intended to "disarm" confounders (confounding variables).

Randomized Controlled Trial (RCT) = Controlled Experiment

List of techniques that help solve intractable inference problems in machine learning that usually arise from interactions between latent variables in a structured graphical model.

-Expectation Maximization

-MAP Inference and Sparse Coding

-Variational Inference and Learning

-Learned Approximate Inference



Approximate Inference 

"The estimand is the target quantity—the purpose of the statistical analysis ... the thing we are estimating" " that connects statistical evidence to theory."

It refers "to both the theoretical and the empirical estimands."

Estimand

1- "Set a **theoretical estimand**, clearly connecting this quantity to theory"

2- "Link to an **empirical estimand**, which is informative about the theoretical estimand under some identification assumptions"

3- Use "an **estimation strategy** to learn the empirical estimand (e.g., a regression model) ... from data."

Three Critical Choices in Causal Inference

- Correlation does not imply causation.
- Causation does not imply correlation either.

Correlation vs. Causation

Establishing causality in economics is a significant challenge for two main reasons. First, unlike in many natural sciences, it is often impossible to conduct controlled experiments to isolate a single variable. The economy is a dynamic system where numerous factors change simultaneously, creating confounding variables that can lead to misleading correlations. Second, the economy's fundamental complexity, arising from the interactions of millions of people, makes it impossible to measure and understand every variable. This inherent complexity means economists must be skeptical of simple claims that one complex phenomenon, like capitalism, is the direct cause of another, such as a rise in living standards. To address these challenges, economists use methods like natural experiments and advanced statistical techniques to infer causal relationships from observational data.

Learn Before

Related

Learn After