A development team is building an AI-powered legal assistant designed to summarize lengthy court transcripts, which often exceed 50,000 words. They are choosing between two pre-trained language models:
- Model A: Achieves state-of-the-art accuracy on summarization tasks up to 2,000 words, but its processing time and computational cost increase exponentially as the input text gets longer.
- Model B: Has slightly lower accuracy on summarization tasks under 2,000 words, but its process
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Performance Enhancement via Long-Context Injection at Inference
A development team is building an AI-powered legal assistant designed to summarize lengthy court transcripts, which often exceed 50,000 words. They are choosing between two pre-trained language models:
- Model A: Achieves state-of-the-art accuracy on summarization tasks up to 2,000 words, but its processing time and computational cost increase exponentially as the input text gets longer.
- Model B: Has slightly lower accuracy on summarization tasks under 2,000 words, but its process
AI Assistant Performance Bottleneck
Prioritizing Computational Efficiency in AI System Design
Inference-Time Scaling