Learn Before
Structured Search in Decoding
An alternative to conventional decoding algorithms involves using compact structures, such as trees or graphs, to represent a wide range of potential outputs. In this approach, each node signifies an intermediate step or a partial solution, while the connections between nodes represent transitions in the reasoning process. This structured method allows the model to explore and evaluate multiple reasoning paths concurrently.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Sampling-Based Search for LLM Inference
Sequence Evaluation using Log-Probability
Deterministic Decoding Algorithms
Modifying the Search Objective to Improve Decoding
Maximum a Posteriori (MAP) Decoding
Speculative Decoding
Structured Search in Decoding
Trade-off between Search Quality and Computational Efficiency in Heuristic Search
An engineer is building a real-time chatbot that must respond to user queries very quickly. To achieve this speed, the engineer implements a text generation strategy that, at each step of forming a response, considers only a small subset of the most likely next words instead of all possible words in the vocabulary. What is the fundamental trade-off inherent in this design choice?
Evaluating a Decoding Algorithm Claim
Analysis of Competing Text Generation Systems
Learn After
Monte Carlo Tree Search-Inspired Decoding
Choosing a Decoding Strategy for a Logic Puzzle AI
A language model is tasked with generating a step-by-step solution to a logic puzzle. The puzzle has several plausible initial moves, but an incorrect choice early on will lead to an unsolvable state later. A simple decoding approach that always commits to the single most probable next step often fails on this task. Which of the following alternative strategies is best designed to overcome this specific challenge?
Advantage of Structured Search in Complex Reasoning