Learn Before
In a Rotary Positional Embedding system, a specific component k uses sine and cosine functions of the position i to create a rotational transformation. For this component, these functions have a period of T. How does the rotational transformation for this component at position i compare to the rotational transformation at position i + T?
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formula for the Period of RoPE's Sine and Cosine Components
In a Rotary Positional Embedding system, a specific component
kuses sine and cosine functions of the positionito create a rotational transformation. For this component, these functions have a period ofT. How does the rotational transformation for this component at positionicompare to the rotational transformation at positioni + T?Analyzing the Role of Period Length in RoPE
Analyzing Positional Encoding in RoPE