Learn Before
In a sequence processing model that uses relative positional information, a query is located at position 5 and a key is located at position 9. What is the calculated offset representing the relative distance from the query to the key?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formula for One-to-One Mapping in T5 Bias Bucketing
In a sequence processing model that uses relative positional information, a query is located at position 5 and a key is located at position 9. What is the calculated offset representing the relative distance from the query to the key?
In a sequence processing model, the relative position between a query at index
iand a key at indexjis calculated as the offseti - j. If the calculated offset for a specific query-key pair is a negative value, what can be inferred about their positions in the sequence?Debugging a Relative Position Calculation