Learn Before
Analyzing Trade-offs in Inference-Time Search Configuration
A developer is using a single, pre-trained language model for two distinct applications: a real-time customer service chatbot where response speed is critical, and a legal document summarizer where accuracy and thoroughness are the top priorities. Analyze how the developer should adjust the inference-time search process for each application. Justify your recommendations by explaining the relationship between the scope of the search and the desired output characteristics.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Benefit of Search Space Expansion in Complex Decoding Tasks
Computational Costs of Search Scaling
Scaling Output Length in Search Scaling
Scaling the Search Space in Search Scaling
An engineer is using a fixed, pre-trained language model to generate a complex travel itinerary. The initial outputs are often functional but fail to find the most optimal route. The engineer cannot alter the model's internal parameters. Which of the following adjustments to the generation process is a direct application of search scaling to find a better itinerary?
Applying Search Scaling Strategies
Analyzing Trade-offs in Inference-Time Search Configuration
Implicit Search Scaling in Search Procedures