Learn Before
Defining the Candidate Set in a Greedy Decoding Strategy
A language model is generating a sequence using a decoding strategy where, at each step, it only pursues the single most likely option. After step i-1, the single best sequence is y_1...y_{i-1}. At step i, the model computes the probabilities for the next token, finding that p(token_A | y_1...y_{i-1}) = 0.6, p(token_B | y_1...y_{i-1}) = 0.3, and p(token_C | y_1...y_{i-1}) = 0.1. Formally define the candidate set of sequences, Y_i, at the end of step i and explain the reasoning behind its composition.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A text generation model employs a decoding strategy where, at each step, it extends the single most probable sequence generated so far. Suppose after three steps, the sequence 'The cat sat' has been generated. In the fourth step, the model calculates probabilities for the next word and finds that 'on' is the most likely, 'under' is the second most likely, and 'by' is the third most likely. Based on this strategy, what is the composition of the candidate set of sequences immediately after this fourth step is completed?
In a text generation model that employs a greedy search decoding strategy, the size of the candidate set of sequences, denoted as
|Y_i|, increases at each stepibecause the model considers a growing number of possible sentence completions.Defining the Candidate Set in a Greedy Decoding Strategy