Learn Before
Modifying the Search Objective to Improve Decoding
A common strategy for improving text generation is to alter the search objective used during the decoding process. Rather than adhering to a single criterion, such as finding the most probable sequence, the goal can be modified to optimize for different qualities in the output.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Sampling-Based Search for LLM Inference
Sequence Evaluation using Log-Probability
Deterministic Decoding Algorithms
Modifying the Search Objective to Improve Decoding
Maximum a Posteriori (MAP) Decoding
Speculative Decoding
Structured Search in Decoding
Trade-off between Search Quality and Computational Efficiency in Heuristic Search
An engineer is building a real-time chatbot that must respond to user queries very quickly. To achieve this speed, the engineer implements a text generation strategy that, at each step of forming a response, considers only a small subset of the most likely next words instead of all possible words in the vocabulary. What is the fundamental trade-off inherent in this design choice?
Evaluating a Decoding Algorithm Claim
Analysis of Competing Text Generation Systems
Learn After
MBR Decoding as an Alternative to MAP Decoding
Incorporating Penalty Terms for Controllable Decoding
Improving Generic Text Generation
A language model is tasked with generating creative story endings. Its current decoding process consistently produces endings that are grammatically perfect and logically sound, but are often predictable and repetitive (e.g., '...and they all lived happily ever after.'). Which of the following statements best analyzes why modifying the search objective could address this issue?
Diagnosing Repetitive Text Generation