Google

An expansion step in a sequence generation algorithm involves generating potential next tokens to extend existing sequences. For example, after processing a certain part of a sequence, the model might identify 'on', 'cute', 'are', and 'sick' as candidate tokens for the next position.

Example of Candidate Tokens in an Expansion Step

A language model is in the process of generating a sentence. It has already produced the partial sequence: 'The fluffy cat sat'. In the next step, the model needs to generate a set of potential words to extend this sequence. Which of the following lists best represents a plausible set of *candidate tokens* for the next position?

A language model is generating text and has produced the sequence: 'The scientist discovered a new...'. In the next step, it considers several potential words to continue the sentence. Analyze the provided list of these candidate tokens and their likelihood scores. Based on the preceding text, explain why 'species' and 'planet' have high scores while 'bicycle' has a very low score.

Analyzing Candidate Token Likelihood

Imagine a language model has generated the partial sentence: 'After the long hike, the explorers were feeling very'. Provide three distinct and plausible words (tokens) that the model might consider as candidates to continue this sentence, and briefly explain why each is a plausible choice.

Learn Before

Related