Formula for the Candidate Set in Top-K Decoding
In top- decoding strategies like beam search, the set of all candidate sequences at step is denoted by . This set is composed of the new sequences that were formed by appending each of the top- next tokens to the shared preceding sequence. The formal definition of this set is:
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
A language model is generating a sequence of tokens. The sequence generated so far is
[501, 243, 988]. At the current step, the model has identified the 3 most probable next tokens as[104, 675, 312]. Based on this information, what is the resulting set of new candidate sequences?Deconstructing Candidate Sequences
Formula for the Candidate Set in Top-K Decoding
A language model is generating text using a top-K decoding strategy. Arrange the following steps in the correct order to describe how a single new candidate sequence is constructed from a given preceding sequence and a set of top-K next tokens.
Learn After
A text generation model has produced the sequence of tokens
('The', 'quick', 'brown'). At the current step, the model calculates the probabilities for the next token and identifies the top 3 most likely tokens as('fox', 'dog', 'wolf'), in that order of probability. Given this information, what is the resulting set of candidate sequences?In a text generation process at step
i, the candidate set is defined as the single most probable sequence formed by appending the most likely next token to the shared preceding sequence.Defining the Candidate Set in Top-K Decoding