Next-Word Prediction Model
A language model is designed to predict the next word in a sentence by choosing the word with the highest probability given the preceding text. Based on this principle, what word will the model predict next, and why?
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Next-Word Prediction Model
A language model's prediction rule is to select the output with the highest conditional probability. Given the input text 'The ocean is deep and...', the model computes the following probabilities for the next word:
- P('mysterious' | 'The ocean is deep and...') = 0.55
- P('blue' | 'The ocean is deep and...') = 0.30
- P('empty' | 'The ocean is deep and...') = 0.10
- P('loud' | 'The ocean is deep and...') = 0.05
Based on its prediction rule, which word will the model choose?
Temperature-Scaled Softmax for Token Probability
Evaluating the 'Arg Max' Prediction Strategy