Formula for the Candidate Set in Greedy Search
In a greedy search decoding algorithm, the set of candidate sequences at step , denoted , is reduced to a singleton set containing only the single most optimal sequence generated up to that point. This sequence is constructed by appending the most probable next token to the previous prefix: . The candidate set is formally expressed as .
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Formula for the Candidate Set in Greedy Search
A text generation model using a greedy decoding approach has produced the sequence 'The dog chased'. At the current step, it calculates the probabilities for the next token as follows: 'the' (0.7), 'a' (0.2), 'its' (0.08), and 'his' (0.02). Based on this information, what constitutes the set of candidate sequences that will be considered for the next step of the generation process?
When generating text by iteratively appending the single most probable token at each step, the collection of partial sequences under consideration for the next step always includes multiple high-probability options.
Candidate Sequence in a Word-by-Word Generation
Formula for the Candidate Set in Greedy Search
A language model is generating text one token at a time by always selecting the single most probable next token. It has already produced the sequence 'The sun is shining'. For the very next step, the model calculates the following conditional probabilities for the next token:
- P(brightly | 'The sun is shining') = 0.55
- P(today | 'The sun is shining') = 0.25
- P(and | 'The sun is shining') = 0.15
- P(down | 'The sun is shining') = 0.05
Based on this method of construction, what will the updated sequence be after this step?
A language model generates text by always appending the single most probable token given the sequence generated so far. Arrange the following steps to correctly illustrate how the model would construct the three-token sequence 'The quick fox'.
Analyzing a Sequence Construction Method
Learn After
A text generation model employs a decoding strategy where, at each step, it extends the single most probable sequence generated so far. Suppose after three steps, the sequence 'The cat sat' has been generated. In the fourth step, the model calculates probabilities for the next word and finds that 'on' is the most likely, 'under' is the second most likely, and 'by' is the third most likely. Based on this strategy, what is the composition of the candidate set of sequences immediately after this fourth step is completed?
In a text generation model that employs a greedy search decoding strategy, the size of the candidate set of sequences, denoted as
|Y_i|, increases at each stepibecause the model considers a growing number of possible sentence completions.Defining the Candidate Set in a Greedy Decoding Strategy