Google

The expansion stage is the first step in a decoding process like 1-best selection. It involves generating a set of potential next tokens, or candidates, based on the preceding context. For instance, as illustrated in the provided diagram, given the token 'is', the expansion stage can produce a candidate set that includes '.', 'cute', 'on', 'are', and 'sick'.

Expansion Stage in 1-Best Selection

A language model is generating text and has so far produced the sequence: 'The quick brown fox jumps over the'. Describe the immediate first action the model takes to determine the very next word, and provide three distinct and plausible examples of words that might be produced during this initial action.

Initial Step in Text Generation

In a multi-stage text generation process, the initial 'expansion' stage is responsible for analyzing the probabilities of all potential next tokens and selecting only the single most probable token to continue the sequence.

A language model is generating text one token at a time using a multi-stage process. After processing the input 'The sky is', the model's first stage is to generate a set of potential next tokens. Which of the following options best represents the direct output of this initial generation stage?

Learn Before

Related