Learn Before
Defining the Candidate Set in Top-K Decoding
A language model is generating a sequence. At step i=4, the shared preceding sequence is ('The', 'cat', 'sat'). The model has identified the top-K next tokens (where K=3) as y_4^top1 = 'on', y_4^top2 = 'by', and y_4^top3 = 'near'. Using standard set notation, write out the complete candidate set, denoted as Y_4.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A text generation model has produced the sequence of tokens
('The', 'quick', 'brown'). At the current step, the model calculates the probabilities for the next token and identifies the top 3 most likely tokens as('fox', 'dog', 'wolf'), in that order of probability. Given this information, what is the resulting set of candidate sequences?In a text generation process at step
i, the candidate set is defined as the single most probable sequence formed by appending the most likely next token to the shared preceding sequence.Defining the Candidate Set in Top-K Decoding