Base Case for Sequence Probability

Joint Probability of a Generated Sequence using the Chain Rule

Relationship Between Joint, Conditional, and Marginal Log-Probabilities of Sequences

Derivation of Sequence Log-Probability via Chain Rule

Logarithmic Form of the Chain Rule for Sequence Probability

Formula for an Impossible Initial Event

A language model is tasked with calculating the total probability of the three-token sequence 'the cat sat'. The model provides the following probability estimates:

• The probability of the first token is Pr("the") = 0.1
• The probability of the second token, given the first, is Pr("cat" | "the") = 0.5
• The probability of the third token, given the first two, is Pr("sat" | "the", "cat") = 0.8

Using the principle that the joint probability of a sequence is the product of the conditional probabilities of its components, what is the joint probability Pr("the", "cat", "sat")?

Computational Stability of Sequence Probability

Which of the following expressions correctly decomposes the joint probability of a four-token sequence (x₁, x₂, x₃, x₄) using the chain rule of probability?