Learn Before
Constructing the Top-k Selection Pool
A language model is at a specific generation step, which we can call step . It has calculated the probabilities for the next token over a small vocabulary: P('on')=0.05, P('the')=0.45, P('a')=0.25, P('in')=0.10, P('at')=0.15. If the model is using a sampling method where the selection pool is limited to the 3 most probable tokens, what is the set for this step? Please write your answer using set notation, e.g., {'token1', 'token2'}.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is generating text and at a certain step, it predicts the following probabilities for the next token from its vocabulary: {'the': 0.4, 'a': 0.2, 'cat': 0.15, 'dog': 0.1, 'ran': 0.08, 'sat': 0.07}. If the model is configured to use a sampling strategy where the next token is chosen from the 3 most probable candidates, which set represents the selection pool for this step?
Constructing the Top-k Selection Pool
If the parameter 'k' is set to 1, the resulting selection pool will contain only the single most probable token, making the sampling process from this pool effectively equivalent to a greedy search at that step.