If two variables are independent, the occurrence of one would not affect another. $P(X, Y) = P(X) \times  P(Y).$ 
If two variables are conditional independent, they are independent given a condition. $P(X, Y | Z) = P(X | Z) \times P(Y | Z).$

Johns Hopkins University

- Combinatorics
- Probability Distributions
- Random Variables
- Independence
- Chain Rules

Topics in Probability Theory

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

Combinatorics is the study of how to count the various combinations, permutations, subgroups, etc. that can be formed from a given set of objects/people/etc. Knowledge of combinatorics is vital for an understanding of probability because when calculating the probability of an event one needs to know the number of potential ways an event can occur. 

Combinatorics

There are 3 possible sources of uncertainty:
- Inherent stochasticity
- Incomplete observability
- Incomplete modeling

Uncertainty 

Common probability distributions include Normal Distribution, Multinoulli Distribution,  Bernoulli Probability Distribution, Exponential Distribution, and Laplace Distribution. 

Common Probability Distributions

This kind of probability deals with the rates at which certain even occur.

Frequentist Probability

When we want to calculate the probability of a subset of variables, we use marginal probability. For discrete variables, marginal probability is calculated by summing over other variables to find the probability of a specific variable. For continuous variables, marginal probability is calculated by performing integration instead of summation. 

Marginal Probability

Conditional probability is the likelihood of an event occurring, given that another event has already occurred. The event that is given must have a non-zero probability. This relationship is commonly denoted as $P(A|B)$ or $Pr(A|B)$, which reads as 'the probability of A given B'. More generally, it can be represented as $Pr(\cdot|x_1, ..., x_n)$ or simply $p(\cdot|x_1, ..., x_n)$, indicating the probability distribution of a variable conditioned on the values of other variables $x_1, ..., x_n$.

Conditional Probability

Any joint probability can be decomposed into conditional probabilities using Chain Rule. For example: 
$$P(a, b, c) = P(a| b, c) \times P(b, c) =  P(a | b, c) \times P(b | c) \times P(c)$$

Chain Rule

Independence and Conditional Independence

This kind of probability deals with the qualitative levels of certainty of various propositions.


Bayesian Probability

A variable that holds different values by chance as a result of a random experiment.

Random Variable

Population distributions are generally used to describe the probability of a random variable or set of random variables taking on each of its possible states.

Learn Before

Related