Learn Before
Selecting a Positional Bias Strategy
Based on the team's observations, would implementing a positional bias using the formula Bias = -βlog(1 + β) (where β is a positive value that increases with token distance) be a suitable strategy? Justify your decision by explaining how the properties of this formula relate to the specific problem the team is facing.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A model designer implements a positional bias using the formula
Bias = -βlog(1 + β), whereβis a positive value that increases with token distance. The goal is to penalize attention to more distant tokens. By mistake, the designer forgets the leading negative sign, implementing the formula asBias = βlog(1 + β). What is the most likely effect of this error on the model's behavior?Analysis of Positional Penalty Growth
Selecting a Positional Bias Strategy