Sparse Attention

Query Prototyping and Memory Compression

Low Rank Self-Attention

Attention with Prior

Improved Multi-Head Attention Mechanism

Linear Attention

A research team is working to reduce the computational cost of the attention mechanism for processing extremely long documents. Their proposed solution involves modifying the attention calculation so that each query token only computes attention scores with a small, fixed subset of key tokens (e.g., neighboring tokens and a few globally important tokens) instead of all tokens in the sequence. Which category of attention improvement best describes this approach?

Match each attention improvement strategy with its core operational principle.

Optimizing Transformer Attention for Long Sequences

Evaluating Attention Optimization Strategies for Specific Applications

Architectural Adaptation of LLMs for Long Sequences

Linear Attention

Classification of Memory Models in LLMs

Memory Models in LLMs as Context Encoders

PagedAttention for KV Cache Memory Optimization

Strategies for Mitigating KV Cache Memory Usage

A machine learning engineer is deploying a large language model and finds that the system frequently runs out of memory during inference. They are investigating two specific high-load scenarios, both of which involve processing a total of 16,000 tokens:

• Scenario X: Processing a batch of 32 user requests simultaneously, where each request has a context length of 500 tokens.
• Scenario Y: Processing a single user request that involves summarizing a very long document with a context length of 16,000 tokens.

Based on how attention states (keys and values) are managed during inference, which statement best analyzes the memory consumption issue?

Architectural Shift in LLMs due to Long-Sequence Limitations

Diagnosing Inference Failures with Long Documents

Analyzing Memory Constraints in Different LLM Applications