An engineer is designing a text-generation model and is considering two different configurations for how each new token attends to previous tokens in the sequence.

• Configuration A: Each new token computes attention scores with only the 16 most recent tokens in the sequence.
• Configuration B: Each new token computes attention scores with all preceding tokens up to a maximum of 512.

Which statement best analyzes the primary trade-off between these two configurations?

Analyzing Sparse Attention Trade-offs

Optimizing a Language Model for Real-Time Translation

In a sparse attention model, expanding the index set G to include more preceding tokens for each query will result in a higher degree of model sparsity and a reduction in computational cost.