Learn Before
Scaling Output Length in Search Scaling
Scaling the output length is a key aspect of search scaling that involves increasing the number of tokens generated during inference. This is crucial for long-form generation tasks like story writing. Additionally, generating outputs with long thinking paths before arriving at final answers significantly enhances complex reasoning, making it highly effective for math problem-solving and code generation. Recent reasoning models, such as OpenAI's o1 and DeepSeek's R1, widely utilize this approach.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Benefit of Search Space Expansion in Complex Decoding Tasks
Computational Costs of Search Scaling
Scaling Output Length in Search Scaling
Scaling the Search Space in Search Scaling
An engineer is using a fixed, pre-trained language model to generate a complex travel itinerary. The initial outputs are often functional but fail to find the most optimal route. The engineer cannot alter the model's internal parameters. Which of the following adjustments to the generation process is a direct application of search scaling to find a better itinerary?
Applying Search Scaling Strategies
Analyzing Trade-offs in Inference-Time Search Configuration
Implicit Search Scaling in Search Procedures
Learn After
Critique of Generation Length Strategy
Improving LLM Summarization Quality
A team is refining a language model's story-generation capabilities. Their primary strategy is to increase the maximum number of tokens the model can produce in a single output, aiming for more comprehensive and detailed narratives. What is the most significant potential downside the team should anticipate as a direct result of only extending the generation length?
Generating and Verifying Thinking Paths