- Discrete Random Variable
- Continous Random Variable

Types of Random Variables

Random variables are typically denoted using a plain typeface lowercase letter and the values it can take on with lowercase script letters.
- The random variable a can have the values of a1 and a2

For vector-valued variables:
- The random variable x can have one of its value as 𝑥

Random Variable Denotation

Sampling is the formal process of acquiring data by drawing independent realizations from an underlying random process. Collecting this empirical data is the necessary first step in using statistical methods to investigate and estimate unknown theoretical probabilities.

Sampling from a Random Process

In probability theory, an event represents a subset of the underlying sample space. When working with a random variable $$X$$, an occurrence where the variable takes on a specific value $$v$$ (denoted by $$X=v$$) or falls within a specific range of values (such as $$1 \leq X \leq 3$$) is considered an event. A probability score, denoted like $$P(X=v)$$ or $$P(1 \leq X \leq 3)$$, is assigned to these events to represent their likelihood.

Probability Event

A variable that holds different values by chance as a result of a random experiment.

Ohio State University, Columbus

- Combinatorics
- Probability Distributions
- Random Variables
- Independence
- Chain Rules

Topics in Probability Theory

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

Combinatorics is the study of how to count the various combinations, permutations, subgroups, etc. that can be formed from a given set of objects/people/etc. Knowledge of combinatorics is vital for an understanding of probability because when calculating the probability of an event one needs to know the number of potential ways an event can occur. 

Combinatorics

There are 3 possible sources of uncertainty:
- Inherent stochasticity
- Incomplete observability
- Incomplete modeling

Uncertainty 

Common probability distributions include Normal Distribution, Multinoulli Distribution,  Bernoulli Probability Distribution, Exponential Distribution, and Laplace Distribution. 

Common Probability Distributions

This kind of probability deals with the rates at which certain even occur.

Frequentist Probability

When we want to calculate the probability of a subset of variables, we use marginal probability. For discrete variables, marginal probability is calculated by summing over other variables to find the probability of a specific variable. For continuous variables, marginal probability is calculated by performing integration instead of summation. 

Marginal Probability

Conditional probability is the likelihood of an event occurring, given that another event has already occurred. The event that is given must have a non-zero probability. This relationship is commonly denoted as $P(A|B)$ or $Pr(A|B)$, which reads as 'the probability of A given B'. More generally, it can be represented as $Pr(\cdot|x_1, ..., x_n)$ or simply $p(\cdot|x_1, ..., x_n)$, indicating the probability distribution of a variable conditioned on the values of other variables $x_1, ..., x_n$.

Conditional Probability

Any joint probability can be decomposed into conditional probabilities using Chain Rule. For example: 
$$P(a, b, c) = P(a| b, c) \times P(b, c) =  P(a | b, c) \times P(b | c) \times P(c)$$

Chain Rule

If two variables are independent, the occurrence of one would not affect another. $P(X, Y) = P(X) \times  P(Y).$ 
If two variables are conditional independent, they are independent given a condition. $P(X, Y | Z) = P(X | Z) \times P(Y | Z).$

Independence and Conditional Independence

This kind of probability deals with the qualitative levels of certainty of various propositions.


Bayesian Probability

Random Variable

Population distributions are generally used to describe the probability of a random variable or set of random variables taking on each of its possible states.

Learn Before

Related

Learn After