Learn Before
Definition

Draft Model Probability Distribution (Prq()Pr_q(\cdot))

A draft model is a smaller, computationally less expensive model used to generate candidate sequences or tokens. The probability distribution Prq()Pr_q(\cdot) represents the likelihood of generating a specific output according to this draft model. This approach is often used in techniques like speculative decoding to accelerate inference in larger, more powerful models by having the large model only verify the draft model's predictions rather than generating tokens from scratch.

0

1

Updated 2025-10-07

Contributors are:

Who are from:

Tags

Ch.5 Inference - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Related