Learn Before
Candidate Set in Greedy Search
In the greedy search decoding algorithm, the set of candidate sequences at step i, denoted Yi, is reduced to contain only a single sequence. This sequence is the one formed by appending the highest-probability token from the current step to the optimal sequence generated in the previous steps.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Mathematical Justification for Greedy Search
Candidate Set in Greedy Search
A language model is generating a two-token sequence. At the first step, it calculates the probability for the next token: 'Token A' has a probability of 0.6, and 'Token B' has a probability of 0.4. If the model chooses 'Token A', the most probable subsequent token is 'Token C' (with a conditional probability of 0.5). If the model had chosen 'Token B', the most probable subsequent token would be 'Token D' (with a conditional probability of 0.9). A text generation algorithm is used that, at every
Algorithm Suitability for Text Generation Tasks
When generating a sequence of text, an algorithm that selects the single most probable token at each step is guaranteed to produce the overall most probable sequence.
Analyzing Suboptimal Outcomes in Text Generation
Selecting and Justifying a Decoding Policy for Two Production Use Cases
Debugging Decoding: Balancing Determinism, Diversity, and Length in a Regulated Product
Post-incident analysis: fixing repetition and truncation by tuning decoding
Choosing a Decoding Configuration Under Latency, Diversity, and Length Constraints
Release-readiness decision: decoding configuration for a customer-facing summarization feature
Decoding policy decision for a multilingual support assistant under safety, latency, and verbosity constraints
You are tuning decoding for an internal "meeting-n...
You’re implementing an LLM feature that generates ...
You’re building an internal “RFP response drafter”...
You’re deploying an LLM to draft customer-facing i...
Greedy Search Strategy in Sequence-to-Sequence Models
Learn After
Formula for the Candidate Set in Greedy Search
A text generation model using a greedy decoding approach has produced the sequence 'The dog chased'. At the current step, it calculates the probabilities for the next token as follows: 'the' (0.7), 'a' (0.2), 'its' (0.08), and 'his' (0.02). Based on this information, what constitutes the set of candidate sequences that will be considered for the next step of the generation process?
When generating text by iteratively appending the single most probable token at each step, the collection of partial sequences under consideration for the next step always includes multiple high-probability options.
Candidate Sequence in a Word-by-Word Generation