Learn Before
Hypothesis in LLM Inference
In the context of Large Language Model (LLM) inference, a hypothesis is formally defined as a tuple consisting of an input sequence and an output sequence . Because the input sequence remains fixed during the inference process, a hypothesis can be simplified to represent only the generated output sequence.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Hypothesis in LLM Inference
Mathematical Formulation of the Search Problem in LLM Inference
Exploration vs. Exploitation in LLM Search
Search Tree Structure in Token Generation
Heuristic Search Algorithms for LLM Inference
Efficient Generation of Candidate Solutions via Search Algorithms
Search for Optimal or Sub-optimal Sequences in LLM Inference
Root of the Search Space as a Representation of Input (x)
A text generation model has a vocabulary of 10,000 possible words it can choose from for each position in a sequence. If this model were to find the optimal output by evaluating every single possible sequence, how would the total number of sequences to check change if the desired output length is increased from 3 words to 5 words?
Evaluating an Inference Strategy
The Impracticality of Exhaustive Search
Historical Context and Computational Challenges of Maximum Probability Prediction
Mathematical Representation of an Output Sequence
Learn After
Search Space in LLM Inference
An LLM is tasked with completing the input sequence
x = 'The best way to learn a new skill is'. During the inference process, a search algorithm considers several potential continuations, such asy1 = 'to practice consistently',y2 = 'by reading a book', andy3 = 'through immersive experience'. How are these potential continuations best described within the context of the search problem?Justification for Hypothesis Simplification
During language model inference, a 'hypothesis' refers exclusively to a potential output sequence, and the input sequence is not considered part of its formal definition.