Formula

Condition for Accepting Speculation (q(y^i+t)p(y^i+t)q(\hat{y}_{i+t}) \le p(\hat{y}_{i+t}))

In speculative decoding, the condition for accepting a speculated token y^i+t\hat{y}_{i+t} outright is met when its probability under the draft distribution, q(y^i+t)q(\hat{y}_{i+t}), is less than or equal to its probability under the target distribution, p(y^i+t)p(\hat{y}_{i+t}). This is expressed as the inequality q(y^i+t)p(y^i+t)q(\hat{y}_{i+t}) \le p(\hat{y}_{i+t}).

Image 0

0

1

Updated 2026-06-18

Contributors are:

Who are from:

Tags

Ch.5 Inference - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Related