Acceptance-Rejection Mechanism for Speculative Decoding

Inequality Constraint for Predicted Future Value Functions ($q(\hat{y}_{i+t}) \leq p(\hat{y}_{i+t})$)

Condition for Rejecting Speculation

Consider a text generation system that uses a fast, approximate model to propose a potential future word. For each proposed word, a more accurate but slower model also calculates a probability. Suppose at a certain step i, the fast model predicts the next word will be 'universe' (represented as $\hat{y}_{i+1}$). The fast model's confidence in this specific prediction is calculated and denoted as $q(\hat{y}_{i+1}) = 0.8$. Based on this information, what is the most accurate interpretation of the value 0.8?

In the context of a system that generates sequences of values (like words in a sentence), the expression $q(\hat{y}_{i+t})$ is often used. Match each component of this expression to its correct description.

Rejection Criterion in Speculative Sampling

Interpreting Model Predictions