Learn Before
Debugging a Relative Position Calculation
Based on the standard formula for calculating the offset between a query at position i and a key at position j, analyze the following scenario, identify the likely error in the calculation, and provide the correct offset value.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formula for One-to-One Mapping in T5 Bias Bucketing
In a sequence processing model that uses relative positional information, a query is located at position 5 and a key is located at position 9. What is the calculated offset representing the relative distance from the query to the key?
In a sequence processing model, the relative position between a query at index
iand a key at indexjis calculated as the offseti - j. If the calculated offset for a specific query-key pair is a negative value, what can be inferred about their positions in the sequence?Debugging a Relative Position Calculation