Learn Before
Speculative Decoding Scenario
Based on the scenario below, determine if the condition for potentially rejecting the proposed word is met. Justify your answer by referencing the relevant probability values.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Probability Formula for Speculation Rejection
An autoregressive model uses two probability distributions, a target distribution 'p' and a proposal distribution 'q', to evaluate potential future values. A condition for potentially rejecting a speculated value is met if its probability under distribution 'q' is greater than its probability under distribution 'p'.
Given the following data for four potential values, which value meets this condition?
Predicted Value Probability under p Probability under q 'A' 0.5 0.4 'B' 0.3 0.6 'C' 0.1 0.1 'D' 0.2 0.05 In a speculative generation process, a model considers a potential future value. This value is evaluated using a target probability distribution,
p, and a proposal probability distribution,q. After evaluation, the potential value is accepted and added to the final output sequence. Based only on the fact that the value was accepted, is the following statement true or false: 'The probability of the value under distributionqmust have been less than or equal to its probability under distributionp.'Speculative Decoding Scenario