Learn Before
Root of the Search Space as a Representation of Input (x)
In the context of LLM inference, where predictions are conditioned on an input sequence x, the starting point or 'root' of the search process can be conceptualized as a representation of this input x. All subsequent predictions branch out from this initial state.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Hypothesis in LLM Inference
Mathematical Formulation of the Search Problem in LLM Inference
Exploration vs. Exploitation in LLM Search
Search Tree Structure in Token Generation
Heuristic Search Algorithms for LLM Inference
Efficient Generation of Candidate Solutions via Search Algorithms
Search for Optimal or Sub-optimal Sequences in LLM Inference
Root of the Search Space as a Representation of Input (x)
A text generation model has a vocabulary of 10,000 possible words it can choose from for each position in a sequence. If this model were to find the optimal output by evaluating every single possible sequence, how would the total number of sequences to check change if the desired output length is increased from 3 words to 5 words?
Evaluating an Inference Strategy
The Impracticality of Exhaustive Search
Historical Context and Computational Challenges of Maximum Probability Prediction
Mathematical Representation of an Output Sequence
Learn After
A language model is tasked with generating a continuation for two different input prompts:
Prompt 1: "The history of the Roman Empire is..." Prompt 2: "The recipe for a perfect pizza dough is..."
When the model begins the process of finding the most likely sequence of words to follow each prompt, how does the starting point (or 'root') of this search process differ between Prompt 1 and Prompt 2?
Analyzing a Faulty Text Generation Process
When a language model is tasked with generating text for two distinct input prompts, the search for the optimal output sequence begins from an identical initial state for both prompts, regardless of the prompts' content.