Euclidean Representation of RoPE Rotation

Element-wise Formula for RoPE Rotation

A positional encoding method can be described in two ways. In one view, a d-dimensional vector is treated as d/2 complex numbers, and each complex number is rotated by an angle. In another view, the same d-dimensional vector is treated as d/2 pairs of real numbers, and each pair is rotated as a 2D vector by the same corresponding angle. What is the core principle that makes these two views equivalent?

Equivalence of 2D Vector Rotation Representations

Verifying Equivalence of 2D Rotational Methods

Euclidean Representation of RoPE Rotation

In a specific positional encoding method, pairs of embedding dimensions are rotated using a 2x2 matrix defined as: $R_{t\theta_k} = \begin{bmatrix} \cos t\theta_k & \sin t\theta_k \\ -\sin t\theta_k & \cos t\theta_k \end{bmatrix}$ where t is the token's position and \theta_k is a frequency. Given a token at position t = 2 and a frequency \theta_k = \pi/4, which matrix correctly represents the rotation?

In a certain positional encoding method, the rotation of a pair of dimensions is achieved using the matrix $R = \begin{bmatrix} \cos(t\theta) & -\sin(t\theta) \\ \sin(t\theta) & \cos(t\theta) \end{bmatrix}$, where t is the position and \theta is a frequency parameter.

Applying a Rotational Transformation Matrix