Complete Sequences as a Stopping Condition for Expansion
In the step-wise expansion of hypotheses during a search, a sequence is considered 'complete' if it terminates with a special end-of-sequence token, such as ⟨EOS⟩. These complete sequences are not expanded further in subsequent steps of the search process.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Complete Sequences as a Stopping Condition for Expansion
Formula for Pruned Step-wise Expansion of the Hypothesis Set
In a text generation process, the set of candidate sequences for the next step is created by appending every word from a fixed vocabulary to the end of each sequence in the current set. If the current set contains 5 candidate sequences and the vocabulary consists of 100 words, how many new candidate sequences will be generated for the next step?
Computational Implications of Hypothesis Expansion
Hypothesis Set Expansion in a Simplified Scenario
Learn After
Notation for the Set of Complete Sequences
In a step-by-step sequence generation process, a set of candidate sequences is maintained and expanded at each step. Suppose at a given step, the current set of candidate sequences is:
["The cat sat", "The dog ran ⟨EOS⟩", "The cat slept on"]Assuming
⟨EOS⟩is a special token indicating the end of a sequence, which of these sequences will be used as a basis for generating longer sequences in the next step?Applying a Stopping Condition in Sequence Expansion
Efficiency of Sequence Expansion