Learn Before
Comparison of Complex and Euclidean Representations of RoPE Rotation
While the formula for Rotary Positional Embeddings (RoPE) in complex space, , appears intricate, it has an equivalent and more intuitive formulation within the d-dimensional Euclidean space.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Consider a 4-dimensional real vector . A transformation is applied that groups the vector's components into consecutive pairs, treats each pair as a complex number (a + ib), and rotates them independently. The first pair (components 1 and 2) is rotated by an angle of radians, and the second pair (components 3 and 4) is rotated by an angle of radians. What is the resulting vector after this transformation?
Determining an Unknown Rotation Angle
A transformation is applied to a d-dimensional real vector by grouping its components into d/2 consecutive pairs. Each pair is treated as representing a point in a 2D plane and is independently rotated by a specific, non-zero angle. True or False: This transformation will always change the Euclidean norm (magnitude) of the original d-dimensional vector.
Learn After
Euclidean Representation of RoPE Rotation
Element-wise Formula for RoPE Rotation
A positional encoding method can be described in two ways. In one view, a d-dimensional vector is treated as d/2 complex numbers, and each complex number is rotated by an angle. In another view, the same d-dimensional vector is treated as d/2 pairs of real numbers, and each pair is rotated as a 2D vector by the same corresponding angle. What is the core principle that makes these two views equivalent?
Equivalence of 2D Vector Rotation Representations
Verifying Equivalence of 2D Rotational Methods
Learn After
Euclidean Representation of RoPE Rotation
Element-wise Formula for RoPE Rotation
A positional encoding method can be described in two ways. In one view, a d-dimensional vector is treated as d/2 complex numbers, and each complex number is rotated by an angle. In another view, the same d-dimensional vector is treated as d/2 pairs of real numbers, and each pair is rotated as a 2D vector by the same corresponding angle. What is the core principle that makes these two views equivalent?
Equivalence of 2D Vector Rotation Representations
Verifying Equivalence of 2D Rotational Methods
Related
Consider a 4-dimensional real vector . A transformation is applied that groups the vector's components into consecutive pairs, treats each pair as a complex number (a + ib), and rotates them independently. The first pair (components 1 and 2) is rotated by an angle of radians, and the second pair (components 3 and 4) is rotated by an angle of radians. What is the resulting vector after this transformation?
Determining an Unknown Rotation Angle
A transformation is applied to a d-dimensional real vector by grouping its components into d/2 consecutive pairs. Each pair is treated as representing a point in a 2D plane and is independently rotated by a specific, non-zero angle. True or False: This transformation will always change the Euclidean norm (magnitude) of the original d-dimensional vector.