Formula for the Search Space as a Union of Complete Sequences
The entire search space, denoted by , is constructed by aggregating all complete sequences generated throughout the step-wise expansion process. It is formally defined as the union of the sets of complete sequences, , from each step up to a maximum sequence length, . The formula is: This expression provides a constructive definition of the search space as the collection of all valid, terminated hypotheses.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Formula for the Search Space as a Union of Complete Sequences
In a sequence generation process, a sequence is considered 'complete' if it ends with a special
⟨EOS⟩token. At a given stepi, the set of all candidate sequences isYi = {('The', 'cat', 'sat'), ('The', 'cat', 'slept', '⟨EOS⟩'), ('The', 'dog'), ('The', 'dog', 'barked', '⟨EOS⟩')}. IfΨ(Yi)represents the subset of sequences inYithat are complete, which of the following correctly identifiesΨ(Yi)?Defining the Set of Complete Sequences
In a step-wise sequence generation process, the set denoted by
Ψ(Yi)contains all candidate sequences from stepithat will be expanded to create new, longer sequences in the subsequent step.Formula for the Search Space as a Union of Complete Sequences
Formula for the Expansion of the Search Space at Each Step
A simplified language model has a vocabulary consisting of only three unique tokens: 'cat', 'sat', and 'on'. The model is configured to generate an output sequence with a fixed length of exactly two tokens. Which of the following options correctly represents the complete set of all possible output sequences the model can generate?
Analyzing Search Space Dimensions
Growth of the Generative Search Space
Mathematical Formulation of the Search Problem in LLM Inference
Learn After
In a step-wise sequence generation process, let
Ψ(Y_i)denote the set of output sequences that are considered complete at stepi. Suppose after running the process for three steps, the following sets of complete sequences have been identified:Ψ(Y_1)= { "The cat." }Ψ(Y_2)= { }Ψ(Y_3)= { "It sat.", "A dog." }
Based on the principle that the total search space
Yis the aggregation of all complete sequences found across all steps, what is the resulting setY?Consider a sequence generation process that runs for a maximum of 5 steps. If a particular sequence is identified as 'complete' at step 2, but the process continues to generate other sequences until step 5, that sequence from step 2 is excluded from the final aggregated search space of all possible complete outputs.
Evaluating a Search Space Construction Method