A language model's attention mechanism uses a relative positional bias. During its training on text segments never exceeding 512 tokens, it learns a unique bias parameter for each specific relative distance from 1 to 63. However, for all distances from 64 to 127, it uses a single shared parameter, and for all distances from 128 to 255, it uses another single shared parameter, and so on. The model is now required to process a document of 2048 tokens. Which statement best analyzes the primary benefit of using shared parameters for larger distances in this scenario?

Model Selection for Long-Sequence Tasks

Rationale for Parameter Sharing in Positional Bias