Learn Before
Predictive Inference by Maximizing Conditional Probability
In probabilistic models, the goal is to determine the best output sequence, , for a given input sequence, . This is accomplished by identifying the possible output sequence, , that has the highest conditional probability. The formal expression for this predictive rule is: This equation states that the prediction is the argument (arg max) that maximizes the conditional probability . The probability distribution is parameterized by the model's weights, denoted by the superscript w.

0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.5 Inference - Foundations of Large Language Models
Ch.2 Generative Models - Foundations of Large Language Models
Related
Predictive Inference by Maximizing Conditional Probability
Predictive Inference in Large Language Models
A predictive model is designed to generate an output, , that minimizes a given loss function, , which measures the error of the prediction. Which of the following expressions correctly represents the formal definition of the model's predicted value, ?
Formal Prediction Expressions for Different Objectives
A predictive model's output, , is formally defined as the argument that optimizes an objective function. Match each modeling objective to the correct formal expression that represents it.
Learn After
Next-Word Prediction Model
A language model's prediction rule is to select the output with the highest conditional probability. Given the input text 'The ocean is deep and...', the model computes the following probabilities for the next word:
- P('mysterious' | 'The ocean is deep and...') = 0.55
- P('blue' | 'The ocean is deep and...') = 0.30
- P('empty' | 'The ocean is deep and...') = 0.10
- P('loud' | 'The ocean is deep and...') = 0.05
Based on its prediction rule, which word will the model choose?
Temperature-Scaled Softmax for Token Probability
Evaluating the 'Arg Max' Prediction Strategy