Learn Before
In an iterative speculative decoding process, the current context is represented by the sequence of tokens {y_1, ..., y_i}. In the current step, two draft tokens are accepted, {ŷ_{i+1}, ŷ_{i+2}}, and the verification model generates one final token, {ȳ_{i+3}}. What will be the complete input context for the next iteration of this process?
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
In an iterative speculative decoding process, the current context is represented by the sequence of tokens
{y_1, ..., y_i}. In the current step, two draft tokens are accepted,{ŷ_{i+1}, ŷ_{i+2}}, and the verification model generates one final token,{ȳ_{i+3}}. What will be the complete input context for the next iteration of this process?A single iteration of the speculative decoding process involves several key actions. Arrange the following actions in the correct chronological order to represent one complete cycle, starting from a given context.
Debugging a Speculative Decoding Implementation