A PMF is used to describe a probability distribution.
- P(𝑥) is used to denote the probability that x = 𝑥
- A probability of 0 means x = 𝑥 is always false
- A probability of 1 means x = 𝑥 is always true

Probability Mass Function (PMF)

- A joint probability distribution is used to describe a probability distribution over various variables. 
- If all possible states are equally likely, we describe it as a uniform distribution.
- The description will depend on whether the distribution consists of discrete or continuous variables.

Probability Distribution Description

A data scientist is modeling the possible outcomes of a single event, which can result in one of six states (State 1 through State 6). Which of the following sets of probabilities represents a valid probability distribution for this event?

A machine learning model is designed to classify an animal image into one of three possible categories: 'Cat', 'Dog', or 'Bird'. For a specific image, the model outputs the following likelihoods for each category: P(Cat) = 0.5, P(Dog) = 0.6, P(Bird) = -0.1. Analyze this output and explain, with reference to the fundamental properties of a probability distribution, why this set of values is invalid.

Validating a Model's Probability Output

An AI model is designed to classify customer feedback into one of three mutually exclusive categories: 'Positive', 'Neutral', or 'Negative'. For a particular piece of feedback, the model outputs the following probabilities:

- P(Positive) = 0.65
- P(Neutral) = 0.25

Due to a software glitch, the probability for the 'Negative' category is missing. Based on the fundamental properties that govern the probabilities of all possible outcomes for a single event, what must be the probability for the 'Negative' category? Explain your reasoning.

AI Sentiment Analysis Model Calibration

Population distributions are generally used to describe the probability of a random variable or set of random variables taking on each of its possible states.

Ohio State University, Columbus

Google

- Combinatorics
- Probability Distributions
- Random Variables
- Independence
- Chain Rules

Topics in Probability Theory

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

Reference of Foundations of Large Language Models Course

Combinatorics is the study of how to count the various combinations, permutations, subgroups, etc. that can be formed from a given set of objects/people/etc. Knowledge of combinatorics is vital for an understanding of probability because when calculating the probability of an event one needs to know the number of potential ways an event can occur. 

Combinatorics

There are 3 possible sources of uncertainty:
- Inherent stochasticity
- Incomplete observability
- Incomplete modeling

Uncertainty 

Common probability distributions include Normal Distribution, Multinoulli Distribution,  Bernoulli Probability Distribution, Exponential Distribution, and Laplace Distribution. 

Common Probability Distributions

This kind of probability deals with the rates at which certain even occur.

Frequentist Probability

When we want to calculate the probability of a subset of variables, we use marginal probability. For discrete variables, marginal probability is calculated by summing over other variables to find the probability of a specific variable. For continuous variables, marginal probability is calculated by performing integration instead of summation. 

Marginal Probability

Conditional probability is the likelihood of an event occurring, given that another event has already occurred. The event that is given must have a non-zero probability. This relationship is commonly denoted as $P(A|B)$ or $Pr(A|B)$, which reads as 'the probability of A given B'. More generally, it can be represented as $Pr(\cdot|x_1, ..., x_n)$ or simply $p(\cdot|x_1, ..., x_n)$, indicating the probability distribution of a variable conditioned on the values of other variables $x_1, ..., x_n$.

Conditional Probability

Any joint probability can be decomposed into conditional probabilities using Chain Rule. For example: 
$$P(a, b, c) = P(a| b, c) \times P(b, c) =  P(a | b, c) \times P(b | c) \times P(c)$$

Chain Rule

This kind of probability deals with the qualitative levels of certainty of various propositions.


Bayesian Probability

A variable that holds different values by chance as a result of a random experiment.

Random Variable

Probability Distribution

If two random variables $$X$$ and $$Y$$ are independent, the occurrence of one does not affect the other: $$P(X, Y) = P(X) \times P(Y)$$ If $$X$$ and $$Y$$ are conditionally independent given a variable $$Z$$, they become independent once the value of $$Z$$ is known: $$P(X, Y | Z) = P(X | Z) \times P(Y | Z)$$

Learn Before

Related

Learn After