Deconstructing Candidate Sequences
At a certain step in text generation, a model produces the following three new candidate sequences: [The, cat, sat, on], [The, cat, sat, by], and [The, cat, sat, near]. Based on these outputs, identify two things: 1) The common prefix sequence that was extended, and 2) The set of the most probable next tokens that were used to create these new sequences.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is generating a sequence of tokens. The sequence generated so far is
[501, 243, 988]. At the current step, the model has identified the 3 most probable next tokens as[104, 675, 312]. Based on this information, what is the resulting set of new candidate sequences?Deconstructing Candidate Sequences
Formula for the Candidate Set in Top-K Decoding
A language model is generating text using a top-K decoding strategy. Arrange the following steps in the correct order to describe how a single new candidate sequence is constructed from a given preceding sequence and a set of top-K next tokens.