A language model is generating text using a top-K decoding strategy. Arrange the following steps in the correct order to describe how a single new candidate sequence is constructed from a given preceding sequence and a set of top-K next tokens.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Comprehension in Revised Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is generating a sequence of tokens. The sequence generated so far is
[501, 243, 988]. At the current step, the model has identified the 3 most probable next tokens as[104, 675, 312]. Based on this information, what is the resulting set of new candidate sequences?Deconstructing Candidate Sequences
Formula for the Candidate Set in Top-K Decoding
A language model is generating text using a top-K decoding strategy. Arrange the following steps in the correct order to describe how a single new candidate sequence is constructed from a given preceding sequence and a set of top-K next tokens.