https://towardsdatascience.com/introduction-to-bayesian-networks-81031eeed94e

Reference for "How Bayesian Networks work"

As defined in "The Book of Why", causal diagrams are Bayesian networks where every arrow indicates either a direct causal relation or the possibility of a direct causal relation in the direction that the arrow is facing. Unlike causal diagrams, Bayesian networks do not have to be based on causal relations.

The meaning of the arrows is a major difference between Bayesian networks and causal diagrams. 

In a probabilistic Bayesian network, the arrows into $Y$ mean that the probability of Y is governed by the conditional probability tables for $Y$, given observations of its parent variables. The same is true for causal Bayesian networks, except that the conditional probability tables specify the probability of $Y$ given interventions on the parent variables. Both models specify probabilities for $Y$, not a specific value of $Y$. 

In a structural causal model, there are no conditional probability tables. The arrows simply mean $Y$ is a function of its parents, as well as the exogenous variable $U_Y$:
$Y = f_Y(X, A, B, C,…, U_Y)$


Relation between Bayesian Networks and Causal Diagrams

This Bayesian network example shows the causal relationships between different seasons (X1), "whether it is raining (X2), whether the sprinkler is on (X3), whether pavement is wet (X4), and whether pavement is slippery (X5)". Since there is no direct link between X1 and X5, we can assume that season does not have a direct causal influence on how slippery the pavement is. Instead, it is influenced by how wet the pavement is (X4).

Bayesian Network Example

A Bayesian network's structure is made up of nodes (which can be multi-variate), edges between paired nodes, and a conditional probability distribution in each nodes. The nodes in a Bayesian network represent random variables and edges represent the conditional dependence of these nodes. We try to use this structure to model the conditional probability distribution of outcome variables (which are often causal) after observing new evidence.

University of Michigan - Ann Arbor

Bayesian networks were developed to understand reasoning under conditions of uncertainty. It is based on Bayes Theorem and is a probabilistic model that represents a set of variables and their conditional dependencies by being able to find the probability of facts being true/false given already defined facts. A Bayesian network is represented by a directed acyclic graph which helps to visualize the probabilistic model for reviewing relationships between random variables and reasonings for causal probabilities given certain facts



Bayesian Networks

Association refers to linking variables together solely based on their perceived outcomes and is not influenced by reasoning. 

Association

Pearl, J., & Mackenzie, D. (2018). $\mathit{The \ Book \ of \ Why: \ The \ New \ Science \ of \ Cause \ and \ Effect.}$ Basic Books.

The Book of Why

How do Bayesian Networks work?

This article covers Bayesian networks and how they work by looking at different aspects of probabilistic graphical models.

[www.bayesserver.com/docs/introduction/bayesian-networks#_dynamic-bayesian-networks](https://www.bayesserver.com/docs/introduction/bayesian-networks#_dynamic-bayesian-networks) 

Helpful article about Bayesian Networks

Belief Propagation is an iterative message passing algorithm used within graphs or trees. In Bayesian Networks, bayesian likelihoods are calculated at certain nodes and passed up or down the network.

* If the message is passed downward (e.g. from parent to child), the child can update its beliefs using conditional probabilities.
* If the message is passed upward (e.g. from child to parent), the parent can update its beliefs by multiplying them by a likelihood ratio.

Belief Propagation in Bayesian Networks

Looking at related information based on the value of one variable is called conditioning on a variable. This is a helpful tool when programming a machine to update its beliefs: it can focus on relevant information while ignoring the rest.

Conditioning on a Variable

The conditional probability table displays conditional probabilities of mutually dependent random variables of a single variable contingent on the others.  Not to be confused with the Contingency Table, which describes the frequencies of data points in the given category. 

Conditional Probability Table

As mentioned before, Bayesian networks encode conditional independencies, i.e. we can use them to determine "independent" events.

Bayesian Networks and Independence

"Bayesian intelligence failed to bridge the gap between artificial and human intelligence. I'm sure you can figure out the missing ingredient: causality"



Criticism of Bayesian Network from the Judea Pearl

Most of the statistical methods are entirely objective, which means reasoning exclusively from data and experiments. But causal analysis and Bayesian analysis are different in that they involves some subjectivity.

Causal analysis requires some subjective commitments based on one's knowledge in the field before the causal digram is drawn and data is explored.

Similarly, Bayesian analysis combines objective evidence with the prior subjective belief.

From objectivity to subjectivity

Given that the guest chooses door 1 and the host opens door 3, we have: 


$P(1=car|3=opened) $

$= \frac{P(3=opened|1=car) *P(1=car)} {p(3=opened)} $

$=\frac{\frac{1}{2}*\frac{1}{3}}{p(3=opened)}$


P(3=opened)  is the probability that the host opens door 3 when guest picked door 1.

$P(3=opened)$ 
$= P(3=opened |1= car)*P(1= car)+P(3=opened|2= car)*P(2 = car)+P(3=opened |3 = car)*P(3 = car)$
$=\frac{1}{2}*\frac{1}{3}+1*\frac{1}{3}+0*\frac{1}{3}$
$=\frac{1}{2}$


$P(1=car|3=opened) $
$=\frac{\frac{1}{6}}{\frac{1}{2}}$
$=\frac{1}{3}$

Hence the possibility of car is behind door 1 is $1/3$, given guest selected door 1 and host opened door 3. 

Bayesian Solution for Monty Hall’s Paradox


- The COVID-19 testings tend to focus on already hospitalized people, not accounting for the people with mild symptoms and the asymptomatics leading to selection bias.  
- The differences in death rates across countries may not be attributed to clinical, demographic, and environmental factors.  
- Together with random sampling/testing, the Bayesian Network model allows causal explanations from data for better predictions.  


Summary of "COVID-19 infection and death rates: the need to incorporate causal explanations for the data and avoid bias in testing"

Structure Learning is a field of machine learning in which data is used to learn the links of a Bayesian network or Dynamic Bayesian network. The two major approaches for structure learning are the score-based approach and the constraint-based approach.

Structure Learning

-Conditional Probability 
- What is the probability of Z happening given the event of X? This is also written as P(Z|X)

-Correlation/Regression 
        -Shows how the data is related or Associated with eachother 

 -Machine Learing and Deep learning occours at this level because they only predict outcomes they dont reason why things are occouring 
 
 -Statistics occurs on the association rung on the ladder (what data tells us about the result)

Association Examples

A real life example of a question that can be answered through association is: "How likely is a customer who bought toothpaste to also buy dental floss?"

What does a symptom tell you about a disease?

What does wearing glasses tell you about a person's eyesight?

Real life example of association questions/application

In the context of Berkeley's admission paradox, Bickel contends that bias is to do with a pattern of association between a particular decision and particular sex of the applicant.  The words "pattern" and "association" are indications that bias belongs to the rung one of the Ladder of Causation, association.  

Bickel's Definition of Bias

A genome-wide association study (GWAS) is a statistical genomic analysis technique to associate specific genetic variations with particular diseases by looking for patterns and frequencies so that certain genetic variations can be identified to associate with and predict disease.  It is important to note that GWAS cannot tell causality since it can only be used to make associations between variables, -hence the first ladder of causation, Association.  

Learn Before

Related

Learn After