Learn Before
Search Space in LLM Inference
The search space in Large Language Model (LLM) inference, denoted by , is the complete set of all possible hypotheses that the model is capable of generating. Because a hypothesis in this context is simplified to mean an output sequence, the search space effectively consists of every possible output sequence.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Search Space in LLM Inference
An LLM is tasked with completing the input sequence
x = 'The best way to learn a new skill is'. During the inference process, a search algorithm considers several potential continuations, such asy1 = 'to practice consistently',y2 = 'by reading a book', andy3 = 'through immersive experience'. How are these potential continuations best described within the context of the search problem?Justification for Hypothesis Simplification
During language model inference, a 'hypothesis' refers exclusively to a potential output sequence, and the input sequence is not considered part of its formal definition.
Learn After
Formula for the Search Space as a Union of Complete Sequences
Formula for the Expansion of the Search Space at Each Step
A simplified language model has a vocabulary consisting of only three unique tokens: 'cat', 'sat', and 'on'. The model is configured to generate an output sequence with a fixed length of exactly two tokens. Which of the following options correctly represents the complete set of all possible output sequences the model can generate?
Analyzing Search Space Dimensions
Growth of the Generative Search Space
Mathematical Formulation of the Search Problem in LLM Inference