Learn Before
Benefit of Search Space Expansion in Complex Decoding Tasks
Expanding the search space during decoding enhances the probability of discovering higher-quality outputs. This is especially valuable in tasks where the optimal solution is not discernible through purely local, step-by-step decisions.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Benefit of Search Space Expansion in Complex Decoding Tasks
Computational Costs of Search Scaling
Scaling Output Length in Search Scaling
Scaling the Search Space in Search Scaling
An engineer is using a fixed, pre-trained language model to generate a complex travel itinerary. The initial outputs are often functional but fail to find the most optimal route. The engineer cannot alter the model's internal parameters. Which of the following adjustments to the generation process is a direct application of search scaling to find a better itinerary?
Applying Search Scaling Strategies
Analyzing Trade-offs in Inference-Time Search Configuration
Implicit Search Scaling in Search Procedures
Learn After
Consider two text generation tasks for a language model. Task A involves summarizing a factual news article about a recent election. Task B involves generating a short, rhyming poem about the four seasons. In which task would significantly expanding the search space during the generation process be more crucial for achieving a high-quality result, and why?
Diagnosing Suboptimal Creative Generation
Constrained Text Generation and Search Strategy