In a speculative decoding step, a verification model p evaluates a sequence of draft tokens {\hat{y}_{i+1}, \hat{y}_{i+2}, \hat{y}_{i+3}} following a confirmed prefix [\mathbf{x}, \mathbf{y}_{\le i}]. The conditional probability for the second draft token, \hat{y}_{i+2}, is calculated as Pr_p(\hat{y}_{i+2} | ____).
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Comprehension in Revised Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
In a speculative decoding process, a verification model
pis evaluating a sequence of three draft tokens,{\hat{y}_{i+1}, \hat{y}_{i+2}, \hat{y}_{i+3}}, that follow an initial prefix[\mathbf{x}, \mathbf{y}_{\le i}]. How does the context used to calculate the conditional probability for the third draft token,\text{Pr}_p(\hat{y}_{i+3}|...), differ from the context used for the first draft token,\text{Pr}_p(\hat{y}_{i+1}|...)?Conditional Probability in Verification
In a speculative decoding step, a verification model
pevaluates a sequence of draft tokens{\hat{y}_{i+1}, \hat{y}_{i+2}, \hat{y}_{i+3}}following a confirmed prefix[\mathbf{x}, \mathbf{y}_{\le i}]. The conditional probability for the second draft token,\hat{y}_{i+2}, is calculated asPr_p(\hat{y}_{i+2} | ____).