Learn Before
Expansion Step in Sequence Generation
In sequence generation algorithms like beam search, the expansion step involves generating a set of new, longer candidate sequences from a set of existing sequences. This is typically done by appending possible next tokens to the end of each existing sequence. This step increases the number of hypotheses under consideration before a selection or pruning step reduces them.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Breadth-First Search for Left-to-Right Token Generation
Root of the Search Tree as Input Representation
Expansion Step in Sequence Generation
In the process of generating an output, a language model considers two different candidate sequences:
Sequence A: 'The quick brown'andSequence B: 'A fast red'. Based on the conceptual model where token generation is a search through a tree, what is the most accurate statement regarding the positions of these two sequences within that tree?In the conceptual search tree for language model output generation, traversing from a parent node to one of its child nodes represents the action of appending a new token to the sequence represented by the parent node.
A language model is generating an output and has produced the following three candidate sequences:
S1: 'The cat sat',S2: 'The cat sat on', andS3: 'The dog ran'. According to the conceptual model of token generation as a search through a tree, which statement accurately describes the relationship between the nodes representing these sequences?Path in Decoding Search Tree as Output Sequence
Learn After
Example of Candidate Tokens in an Expansion Step
Formula for Step-wise Expansion of the Hypothesis Set
A text generation process is at a stage where it has identified 3 distinct, promising sequences, each 4 tokens long. To find the best 5-token sequences, it performs an expansion step where it considers every possible next token for each of the 3 sequences. If the model's vocabulary consists of 30,000 unique tokens, how many new candidate sequences, each 5 tokens long, are created in this single expansion step before any form of filtering or selection is applied?
In a text generation algorithm, the process often involves an 'expansion' step where a set of promising partial sentences are extended by one more word. This is followed by a 'selection' step that keeps only a limited number of the newly formed, longer sentences. What is the primary problem created by the expansion step that necessitates the subsequent selection step?
Identifying the Expansion Phase in Text Generation