Learn Before
Analyzing the Role of Period Length in RoPE
Consider two different components of a Rotary Positional Embedding. Component A is based on sine and cosine functions with a very long period, while Component B uses functions with a very short period. Analyze how the positional information encoded by Component A would differ from that encoded by Component B, and explain the significance of having this variation.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formula for the Period of RoPE's Sine and Cosine Components
In a Rotary Positional Embedding system, a specific component
kuses sine and cosine functions of the positionito create a rotational transformation. For this component, these functions have a period ofT. How does the rotational transformation for this component at positionicompare to the rotational transformation at positioni + T?Analyzing the Role of Period Length in RoPE
Analyzing Positional Encoding in RoPE