Google

When applying a trained language model, a common and fundamental task is next token prediction, which involves finding the most likely token given its sequence of previous context tokens. At each step, the model computes a probability distribution over the entire vocabulary conditioned on the preceding context. This token prediction task sequentially utilizes these computed probability distributions to select the most probable next token and continue the sequence.

Next Token Prediction Task

In the task of next token prediction, a language model determines the most likely subsequent token, $$\hat{x}_i$$, given a preceding context $$x_0,...,x_{i-1}$$. This is achieved by selecting the token from the entire vocabulary $$\mathcal{V}$$ that maximizes the conditional probability output by the model. This selection process is formally expressed as:

$$\hat{x}_i = \argmax_{x_i \in \mathcal{V}} \Pr(x_{i}|x_0,...,x_{i-1})$$

Argmax Formula for Next Token Prediction

An autoregressive language model has just generated the text: 'The sun is shining and the sky is'. Based on the fundamental principle of its operation, what is the immediate next computational step the model must perform to determine the following word?

An autoregressive language model is given the input sequence: 'The cat sat on the'. To determine the next token, the model performs a specific computation. Describe the direct result of this computation. Your description should specify what kind of information is produced and the scope of possibilities it covers.

Characterizing Model Output for Next Token Prediction

An autoregressive language model is given the input sequence 'The cat sat on the'. Arrange the following steps in the correct chronological order that the model follows to generate the very next token.

Learn Before

Related