Learn Before
Computational Infeasibility of Exhaustive Search in LLM Decoding
The complete search space for potential output sequences in LLMs grows exponentially with the length of the generated text. This rapid expansion makes a direct or exhaustive search for the optimal sequence computationally infeasible for all but the shortest outputs.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Heuristic Search Algorithms for LLM Inference
Stopping Criteria in LLM Inference
Computational Infeasibility of Exhaustive Search in LLM Decoding
A language model is given the prompt 'The capital of France is'. Internally, the model's calculations show that the single most probable next word is 'Paris'. However, the model ultimately generates the sequence 'The capital of France is a beautiful city'. Which statement best analyzes the reason for this discrepancy?
The Challenge of Generating Optimal Text
Analyzing Text Generation Behavior
Learn After
Search Space Pruning in LLM Decoding
A language model with a vocabulary of 30,000 unique tokens is generating a response. If the model were to perform a complete, exhaustive search to find the absolute best possible 5-token sequence, which calculation represents the total number of unique sequences it would need to evaluate?
Evaluating a Decoding Strategy Proposal
Decoding Strategy Post-Mortem