In a system that uses a faster, smaller model to generate candidate tokens for a larger, more accurate model, a single token is being evaluated. The faster model assigns a probability of 0.8 to this token, while the more accurate model assigns it a probability of 0.6. For the acceptance check, a random number of 0.7 is drawn from a uniform distribution between 0 and 1. Based on this information, what is the outcome for this candidate token?
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formula for the Number of Consecutively Accepted Tokens in Speculative Decoding
In a system that uses a faster, smaller model to generate candidate tokens for a larger, more accurate model, a single token is being evaluated. The faster model assigns a probability of 0.8 to this token, while the more accurate model assigns it a probability of 0.6. For the acceptance check, a random number of 0.7 is drawn from a uniform distribution between 0 and 1. Based on this information, what is the outcome for this candidate token?
Speculative Decoding Acceptance Analysis
The Role of Randomness in Token Acceptance