Learn Before
Probability Formula for Speculation Rejection
If the condition is met, the associated speculation is rejected with a probability calculated by the formula $1 - \frac{p(\hat{y}{i+t})}{q(\hat{y}{i+t})}$.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Probability Formula for Speculation Rejection
An autoregressive model uses two probability distributions, a target distribution 'p' and a proposal distribution 'q', to evaluate potential future values. A condition for potentially rejecting a speculated value is met if its probability under distribution 'q' is greater than its probability under distribution 'p'.
Given the following data for four potential values, which value meets this condition?
Predicted Value Probability under p Probability under q 'A' 0.5 0.4 'B' 0.3 0.6 'C' 0.1 0.1 'D' 0.2 0.05 In a speculative generation process, a model considers a potential future value. This value is evaluated using a target probability distribution,
p, and a proposal probability distribution,q. After evaluation, the potential value is accepted and added to the final output sequence. Based only on the fact that the value was accepted, is the following statement true or false: 'The probability of the value under distributionqmust have been less than or equal to its probability under distributionp.'Speculative Decoding Scenario
Learn After
A predictive system is evaluating a potential output. The probability of this output under distribution 'p' is 0.2, and the probability under distribution 'q' is 0.8. Given that the probability from 'q' is greater than the probability from 'p', what is the probability that this potential output will be rejected?
Speculative Rejection Scenario
Analyzing the Speculation Rejection Formula