Learn Before
In a speculative generation process, a draft system proposes a sequence of five candidate tokens. A target system validates them one by one. The first token is accepted. The second is accepted. The third is rejected. The fourth is accepted. The fifth is accepted. Based on this outcome, the resulting set of accepted speculative tokens is {token_1, token_2, token_4, token_5}.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
In a text generation process, a draft system proposes a sequence of candidate tokens. A more powerful system then validates them. A token is accepted if a corresponding random number (between 0 and 1) is less than or equal to the ratio of the powerful system's probability to the draft system's probability for that token. The validation process stops immediately after the first token is rejected. Given the data below, what is the resulting set of consecutively accepted tokens?
Token Powerful System Probability Draft System Probability Random Number 'the' 0.8 0.9 0.7 'cat' 0.7 0.8 0.8 'sat' 0.5 0.9 0.6 'on' 0.9 0.95 0.2 Analyzing a Speculative Decoding Outcome
In a speculative generation process, a draft system proposes a sequence of five candidate tokens. A target system validates them one by one. The first token is accepted. The second is accepted. The third is rejected. The fourth is accepted. The fifth is accepted. Based on this outcome, the resulting set of accepted speculative tokens is {token_1, token_2, token_4, token_5}.