A text generation process uses a fast 'draft' model to propose a sequence of tokens and a more powerful 'verification' model to check them. In one step, the draft model proposes the five-token sequence: ['the', 'quick', 'brown', 'fox', 'jumps']. The verification model accepts the first three tokens ('the', 'quick', 'brown') but rejects the fourth token ('fox'). The verification model then generates its own token, 'sly'. What is the complete set of new tokens added to the main sequence in this single step?
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A text generation process uses a fast 'draft' model to propose a sequence of tokens and a more powerful 'verification' model to check them. In one step, the draft model proposes the five-token sequence:
['the', 'quick', 'brown', 'fox', 'jumps']. The verification model accepts the first three tokens ('the','quick','brown') but rejects the fourth token ('fox'). The verification model then generates its own token,'sly'. What is the complete set of new tokens added to the main sequence in this single step?Analysis of a Speculative Generation Step
Iterative Process of Speculative Decoding
In a text generation system using a fast draft model and a more powerful verification model, a single generation step adds the following set of new tokens to the sequence:
{'and', 'the', 'lion'}. Based on the principles of this generation method, which of the following scenarios is the only one that could have produced this specific output?