Learn Before
Example of Candidate Tokens in an Expansion Step
An expansion step in a sequence generation algorithm involves generating potential next tokens to extend existing sequences. For example, after processing a certain part of a sequence, the model might identify 'on', 'cute', 'are', and 'sick' as candidate tokens for the next position.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Example of Candidate Tokens in an Expansion Step
Formula for Step-wise Expansion of the Hypothesis Set
A text generation process is at a stage where it has identified 3 distinct, promising sequences, each 4 tokens long. To find the best 5-token sequences, it performs an expansion step where it considers every possible next token for each of the 3 sequences. If the model's vocabulary consists of 30,000 unique tokens, how many new candidate sequences, each 5 tokens long, are created in this single expansion step before any form of filtering or selection is applied?
In a text generation algorithm, the process often involves an 'expansion' step where a set of promising partial sentences are extended by one more word. This is followed by a 'selection' step that keeps only a limited number of the newly formed, longer sentences. What is the primary problem created by the expansion step that necessitates the subsequent selection step?
Identifying the Expansion Phase in Text Generation
Learn After
A language model is in the process of generating a sentence. It has already produced the partial sequence: 'The fluffy cat sat'. In the next step, the model needs to generate a set of potential words to extend this sequence. Which of the following lists best represents a plausible set of candidate tokens for the next position?
Analyzing Candidate Token Likelihood
Generating Plausible Next Tokens