Learn Before
Computational Costs of Search Scaling
A primary drawback of search scaling is the inherent increase in computational overhead. Expanding the search space, for example by increasing the beam width in beam search, directly results in higher memory usage and extended inference times.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Benefit of Search Space Expansion in Complex Decoding Tasks
Computational Costs of Search Scaling
Scaling Output Length in Search Scaling
Scaling the Search Space in Search Scaling
An engineer is using a fixed, pre-trained language model to generate a complex travel itinerary. The initial outputs are often functional but fail to find the most optimal route. The engineer cannot alter the model's internal parameters. Which of the following adjustments to the generation process is a direct application of search scaling to find a better itinerary?
Applying Search Scaling Strategies
Analyzing Trade-offs in Inference-Time Search Configuration
Implicit Search Scaling in Search Procedures
Learn After
Diminishing Returns in Search Scaling
Optimizing Inference Performance
An engineer modifies a language model's inference procedure to evaluate a significantly larger number of potential output sequences at each generation step, aiming to enhance the final output quality. What is the most direct and unavoidable trade-off associated with this modification?
Resource Consumption in Text Generation