Base Case for Sequence Probability

Joint Probability of a Generated Sequence using the Chain Rule

Relationship Between Joint, Conditional, and Marginal Log-Probabilities of Sequences

Derivation of Sequence Log-Probability via Chain Rule

Logarithmic Form of the Chain Rule for Sequence Probability

Formula for an Impossible Initial Event

A language model is tasked with calculating the total probability of the three-token sequence 'the cat sat'. The model provides the following probability estimates:

• The probability of the first token is Pr("the") = 0.1
• The probability of the second token, given the first, is Pr("cat" | "the") = 0.5
• The probability of the third token, given the first two, is Pr("sat" | "the", "cat") = 0.8

Using the principle that the joint probability of a sequence is the product of the conditional probabilities of its components, what is the joint probability Pr("the", "cat", "sat")?

Computational Stability of Sequence Probability

Which of the following expressions correctly decomposes the joint probability of a four-token sequence (x₁, x₂, x₃, x₄) using the chain rule of probability?

A language model is designed to calculate the probability of a long sentence by sequentially multiplying the conditional probabilities of each word. Each individual word probability is a small floating-point number (e.g., 0.05, 0.1, 0.02). During testing on sentences with over 100 words, the model consistently outputs a final probability of 0.0, even though no single word has a probability of zero. What is the most likely technical reason for this incorrect result?

Comparing Sequence Probabilities in Log Space

Evaluating Sequence Likelihood with Log Probabilities

Logarithmic Form of the Chain Rule for Sequence Probability

Derivation of Sequence Log-Probability via Chain Rule

Sequence Evaluation using Log-Probability