A language model's output probabilities for the next token are sorted in descending order. The candidate pool for sampling, represented as , is constructed by including all tokens whose individual probability is greater than the sampling threshold .
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
At a specific step 'i' in a text generation process, the model has calculated the following probabilities for the next token from a vocabulary of {A, B, C, D, E}:
P(A) = 0.40 P(B) = 0.30 P(C) = 0.15 P(D) = 0.10 P(E) = 0.05
If the sampling process uses a probability threshold 'p' of 0.8, which of the following sets correctly represents the candidate pool of tokens, denoted as ?
Constructing the Top-p Candidate Pool
A language model's output probabilities for the next token are sorted in descending order. The candidate pool for sampling, represented as , is constructed by including all tokens whose individual probability is greater than the sampling threshold .