Learn Before
A language model is generating text and has calculated the following probabilities for the next potential token: {'the': 0.45, 'a': 0.20, 'cat': 0.12, 'dog': 0.08, 'ran': 0.07, 'jumped': 0.05}. If the model is configured to sample its next choice from only the 4 most likely candidates, which set of tokens constitutes the selection pool?
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formal Derivation of the Top-k Selection Pool
A language model is generating text and has calculated the following probabilities for the next potential token:
{'the': 0.45, 'a': 0.20, 'cat': 0.12, 'dog': 0.08, 'ran': 0.07, 'jumped': 0.05}. If the model is configured to sample its next choice from only the 4 most likely candidates, which set of tokens constitutes the selection pool?Impact of Selection Pool Size on Text Generation
When generating text by sampling from a pool of the most probable candidate tokens, setting the pool size to 1 will produce the exact same output sequence as a method that always deterministically chooses the single token with the highest probability at every step.