Learn Before
Analyzing Speculative Sampling Acceptance
An engineer is analyzing a text generation system that uses a small 'draft' model and a large 'target' model. The system's efficiency relies on some tokens being accepted immediately while others undergo a probabilistic check. Given the data below for four consecutively proposed tokens, identify which token is guaranteed to be accepted without a probabilistic rejection check and explain the underlying principle for its acceptance.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
In a text generation process, a small 'draft' model proposes the next token, 'evolution'. The probability assigned to this token by the draft model is 0.8. A larger, more accurate 'target' model then evaluates the same token and assigns it a probability of 0.6. Based on this information, what is the probability that the token 'evolution' will be rejected?
In a speculative sampling process, a draft model proposes four potential next tokens. A more powerful target model then evaluates the probability of each token. Given the probabilities below, which token has the highest probability of being rejected?
Analyzing Speculative Sampling Acceptance