Example of Candidate Tokens in an Expansion Step

Formula for Step-wise Expansion of the Hypothesis Set

A text generation process is at a stage where it has identified 3 distinct, promising sequences, each 4 tokens long. To find the best 5-token sequences, it performs an expansion step where it considers every possible next token for each of the 3 sequences. If the model's vocabulary consists of 30,000 unique tokens, how many new candidate sequences, each 5 tokens long, are created in this single expansion step before any form of filtering or selection is applied?

In a text generation algorithm, the process often involves an 'expansion' step where a set of promising partial sentences are extended by one more word. This is followed by a 'selection' step that keeps only a limited number of the newly formed, longer sentences. What is the primary problem created by the expansion step that necessitates the subsequent selection step?

Identifying the Expansion Phase in Text Generation