Fixed-Size KV Cache for Long-Context Inference

A development team is building a language model designed to summarize entire research books. They find that while the model works well on short chapters, it consistently fails during processing of the full book, citing 'out-of-memory' errors and exhibiting processing times that increase exponentially with the number of pages. Which of the following best identifies the core technical bottleneck and the most relevant class of solutions to explore?

A team of engineers is working to enhance a Large Language Model's ability to process very long documents. They are considering several distinct technical approaches. Match each technical approach with the specific problem it is designed to solve within the context of long-input adaptation.

Evaluating a Long-Input Strategy for a Legal AI