Learn Before
Characterizing Model Output for Next Token Prediction
An autoregressive language model is given the input sequence: 'The cat sat on the'. To determine the next token, the model performs a specific computation. Describe the direct result of this computation. Your description should specify what kind of information is produced and the scope of possibilities it covers.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Argmax Formula for Next Token Prediction
An autoregressive language model has just generated the text: 'The sun is shining and the sky is'. Based on the fundamental principle of its operation, what is the immediate next computational step the model must perform to determine the following word?
Characterizing Model Output for Next Token Prediction
An autoregressive language model is given the input sequence 'The cat sat on the'. Arrange the following steps in the correct chronological order that the model follows to generate the very next token.