Learn Before
Trade-offs in Long-Context Model Selection
A software development company is building a new feature that uses a large language model to analyze and answer questions about entire legal contracts, which can be hundreds of pages long. They are considering two different models. Model A has an extremely large context window, capable of holding the entire contract, but is very slow and expensive to run. Model B has a much smaller context window, is faster and cheaper, but would require the contract to be broken into chunks. Analyze the trade-offs between these two approaches, specifically explaining which challenges of long-context processing each model attempts to mitigate and which challenges it might introduce or exacerbate.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.5 Inference - Foundations of Large Language Models
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Diagnosing Performance Issues in Long-Document Summarization
A research team uses a language model to perform question-answering on a 200-page technical manual. They observe that the model consistently provides accurate answers for questions related to content in the first 10 pages and the last 10 pages, but frequently hallucinates or provides incorrect answers for questions about content from pages 90-110. Which of the following challenges of processing long inputs best explains this specific pattern of failure?
Trade-offs in Long-Context Model Selection
Strategic Information Management in Context Scaling