Short Answer

Analysis of Positional Penalty Growth

A positional bias mechanism uses the formula Bias = -βlog(1 + β) to penalize attention between tokens, where β is a positive value that increases with the distance between the tokens. Describe how the rate of increase of this penalty changes as the distance between tokens grows larger. In other words, is the additional penalty for one extra unit of distance greater for nearby tokens or for very distant tokens? Justify your answer based on the properties of the function.

0

1

Updated 2025-10-04

Contributors are:

Who are from:

Tags

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Computing Sciences

Foundations of Large Language Models Course

Analysis in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science