Learn Before
Analyzing Token Selection Strategies
A language model is given the prompt: 'The old pirate captain stared at the treasure map and muttered, 'The treasure is buried under the giant...''. For the very next token, the model computes the following probability distribution: {'palm': 0.60, 'oak': 0.25, 'crystal': 0.10, 'waterfall': 0.05}. The model is run twice, producing two different completions. Analyze the two outcomes below and explain the likely difference in the token selection process for each run.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Next Token Prediction Task
Token Sampling from a Conditional Probability Distribution
Using Temperature with Softmax to Control Randomness in Token Selection
A language model is generating text and has produced the sequence 'The sky is'. It then calculates the following probability distribution for the next potential token:
{'blue': 0.75, 'green': 0.15, 'bright': 0.08, 'falling': 0.02}. If the model is configured to always select the single token with the highest probability, which token will it choose next?Analyzing Token Selection Strategies
A language model is generating text and encounters the same input sequence on two separate occasions, producing two different probability distributions for the next token, shown below.
- Distribution A:
{'meal': 0.90, 'dish': 0.05, 'surprise': 0.03, 'error': 0.02} - Distribution B:
{'soup': 0.30, 'stew': 0.25, 'salad': 0.22, 'dessert': 0.23}
Which of the following statements provides the most accurate analysis of these two distributions regarding the token selection process?
- Distribution A:
To ensure the generated text is as coherent and factually accurate as possible, a language model must always select the single token with the highest probability from the distribution at each step of the generation process.