Set of Accepted Speculative Tokens

Calculating Consecutively Accepted Tokens

In a speculative decoding process, the number of consecutively accepted tokens from the start of a draft sequence, denoted by $n_a$, is determined by finding the index of the first rejected token. The formula is: $n_a = \min \{t-1 | 1 \le t \le \tau, r_t > \frac{p(\hat{y}_{i+t})}{q(\hat{y}_{i+t})} \}$.

Given a draft sequence of 5 tokens ($\tau=5$) with the following randomly generated numbers ($r_t$) and probability ratios ($p/q$), what is the calculated value of $n_a$?

• t=1: $r_1$=0.4, $p/q$=0.7
• t=2: $r_2$=0.8, $p/q$=0.9
• t=3: $r_3$=0.6, $p/q$=0.5
• t=4: $r_4$=0.3, $p/q$=0.8
• t=5: $r_5$=0.7, $p/q$=0.6

Consider the formula for calculating the number of consecutively accepted tokens in speculative decoding: $n_a = \min \{t-1 | 1 \le t \le \tau, r_t > \frac{p(\hat{y}_{i+t})}{q(\hat{y}_{i+t})} \}$. If the very first token in a drafted sequence (at index $t=1$) is rejected, this formula will yield a value of $n_a = 0$.