Combinatorics is the study of how to count the various combinations, permutations, subgroups, etc. that can be formed from a given set of objects/people/etc. Knowledge of combinatorics is vital for an understanding of probability because when calculating the probability of an event one needs to know the number of potential ways an event can occur. 

Combinatorics

There are 3 possible sources of uncertainty:
- Inherent stochasticity
- Incomplete observability
- Incomplete modeling

Uncertainty 

Common probability distributions include Normal Distribution, Multinoulli Distribution,  Bernoulli Probability Distribution, Exponential Distribution, and Laplace Distribution. 

Common Probability Distributions

This kind of probability deals with the rates at which certain even occur.

Frequentist Probability

When we want to calculate the probability of a subset of variables, we use marginal probability. For discrete variables, marginal probability is calculated by summing over other variables to find the probability of a specific variable. For continuous variables, marginal probability is calculated by performing integration instead of summation. 

Marginal Probability

Conditional probability is the likelihood of an event occurring, given that another event has already occurred. The event that is given must have a non-zero probability. This relationship is commonly denoted as $P(A|B)$ or $Pr(A|B)$, which reads as 'the probability of A given B'. More generally, it can be represented as $Pr(\cdot|x_1, ..., x_n)$ or simply $p(\cdot|x_1, ..., x_n)$, indicating the probability distribution of a variable conditioned on the values of other variables $x_1, ..., x_n$.

Conditional Probability

Any joint probability can be decomposed into conditional probabilities using Chain Rule. For example: 
$$P(a, b, c) = P(a| b, c) \times P(b, c) =  P(a | b, c) \times P(b | c) \times P(c)$$

Chain Rule

If two variables are independent, the occurrence of one would not affect another. $P(X, Y) = P(X) \times  P(Y).$ 
If two variables are conditional independent, they are independent given a condition. $P(X, Y | Z) = P(X | Z) \times P(Y | Z).$

Independence and Conditional Independence

This kind of probability deals with the qualitative levels of certainty of various propositions.


Bayesian Probability

A variable that holds different values by chance as a result of a random experiment.

Random Variable

Population distributions are generally used to describe the probability of a random variable or set of random variables taking on each of its possible states.

Probability Distribution

- Combinatorics
- Probability Distributions
- Random Variables
- Independence
- Chain Rules

Johns Hopkins University

University of Texas at Austin

Probability is the branch of math concerned with the likelihood of an event occuring, or the likelihood that an event is true. 

Probability Theory

- https://en.wikipedia.org/wiki/Probability#:~:text=Probability%20is%20the%20branch%20of,event%20and%201%20indicates%20certainty.
- Morin, D. (2016). Probability for the Enthusiastic Beginner. Las Vegas, Nevada: CreateSpace Independent Publishing Platform.
- https://en.wikipedia.org/wiki/Permutation

Probability References

Topics in Probability Theory

This example demonstrates how to calculate the probability of a specific outcome from multiple independent events. Consider a scenario where three friends each flip a fair coin once, and the question is: what is the probability that exactly one of them flips heads? 

To solve this, we first identify the sample space, which consists of all possible outcomes. Since each of the three flips has two outcomes (Heads or Tails), there are $2^3 = 8$ total possibilities: HHH, HHT, HTH, HTT, THH, THT, TTH, TTT. 

Next, we identify the favorable outcomes that match the condition of having exactly one head: HTT, THT, and TTH. There are 3 such outcomes. 

The probability is the ratio of favorable outcomes to the total number of outcomes, which is 3/8.

Calculating the Probability of Exactly One Head in Three Coin Flips

The binomial probability formula is used to determine the probability of obtaining exactly *k* successes in a sequence of *n* independent trials. The formula is expressed as $\binom{n}{k} p^k (1-p)^{n-k}$, where *p* represents the probability of success on a single trial, and the binomial coefficient $\binom{n}{k}$ calculates the number of ways to choose *k* successes from *n* trials.

Binomial Probability Formula

A person rolls two standard, fair six-sided dice at the same time. What is the probability that the sum of the numbers shown on the top faces is exactly 5?

Based on your understanding of probability, analyze your friend's claim and conclusion. Is this single experiment of 10 flips sufficient evidence to conclude the coin is not a standard, fair coin? Explain your reasoning.

Evaluating a Claim about a 'Lucky' Coin

A fair spinner is divided into four equal sections, colored Red, Blue, Green, and Yellow. If the spinner is spun three independent times, what is the probability that it lands on 'Red' exactly twice? Explain your reasoning by identifying the total number of possible outcomes and the number of favorable outcomes.

Calculating Probability of Compound Events

Surprisal quantifies the level of surprise experienced by an observer upon witnessing an event $$j$$ to which they had assigned a subjective probability $$P(j)$$. It is defined mathematically as:
$$-\log P(j)$$
Events with lower probabilities yield higher surprisal, reflecting that they convey more information when they occur.

Learn Before

Related

Learn After