Learn Before
Trade-offs in Cost-Based Text Generation
A financial services company uses a large language model to generate summaries of lengthy market analysis reports. Each word generated incurs a direct computational cost. Describe how a cost-based stopping criterion could be implemented in this scenario and explain the primary trade-off the company would face by using it.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Evaluating Stopping Criteria for a Time-Sensitive Application
An engineering team is deploying a large language model for a live customer support chatbot. The primary business requirement is to ensure that no user waits more than two seconds for an initial response, even if it means the response is slightly incomplete. Which of the following rules for ending the text generation process is best aligned with this requirement?
Trade-offs in Cost-Based Text Generation