Applying Token Sampling in Text Generation
Based on the provided probability distribution, describe the essential next step the model must take to continue generating the sentence. Explain what this step accomplishes, identify the token most likely to be chosen, and clarify why its selection is not guaranteed.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formula for Token Sampling in Autoregressive Models
Applying Token Sampling in Text Generation
An autoregressive language model has processed the input sequence 'The cat sat on the' and has calculated the following conditional probability distribution for the next token: P('mat'|context) = 0.6, P('rug'|context) = 0.3, P('floor'|context) = 0.08, P('sky'|context) = 0.02. If the model then samples a token from this distribution, which of the following statements is most accurate?
In autoregressive text generation, after the model computes the conditional probability distribution for the next token, the sampling process always selects the token with the highest probability score.