A model designer implements a positional bias using the formula Bias = -βlog(1 + β), where β is a positive value that increases with token distance. The goal is to penalize attention to more distant tokens. By mistake, the designer forgets the leading negative sign, implementing the formula as Bias = βlog(1 + β). What is the most likely effect of this error on the model's behavior?

Analysis of Positional Penalty Growth

Selecting a Positional Bias Strategy