Learn Before
Candidate Set Composition in Sampling-Based Decoding
In a text generation process that uses a sampling-based decoding method, after a single token is sampled and appended to the current sequence at a given step, describe the composition of the set of candidate sequences that will be considered for the next step. Explain the reasoning behind this composition.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
During an autoregressive text generation process using a sampling-based method, the model has produced the sequence 'The sun is shining and the'. At the current step, the token 'sky' is sampled from the model's output distribution. Based on this single event, what is the complete set of candidate sequences that will be used as the basis for generating the next token?
In an autoregressive text generation process where a single token is chosen by sampling from a probability distribution at each step, the algorithm must keep track of several competing sequences simultaneously to decide which one to extend in the next step.
Candidate Set Composition in Sampling-Based Decoding