Essay

Trade-offs in Long-Context Model Selection

A software development company is building a new feature that uses a large language model to analyze and answer questions about entire legal contracts, which can be hundreds of pages long. They are considering two different models. Model A has an extremely large context window, capable of holding the entire contract, but is very slow and expensive to run. Model B has a much smaller context window, is faster and cheaper, but would require the contract to be broken into chunks. Analyze the trade-offs between these two approaches, specifically explaining which challenges of long-context processing each model attempts to mitigate and which challenges it might introduce or exacerbate.

0

1

Updated 2025-10-06

Contributors are:

Who are from:

Tags

Ch.3 Prompting - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Ch.5 Inference - Foundations of Large Language Models

Analysis in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science