Definition of the RoPE Cosine Vector

Definition of the RoPE Sine Vector

Periodicity of RoPE's Sine and Cosine Components

The application of a rotary positional embedding to a two-dimensional vector $(x_1, x_2)$ at position $i$ with frequency $\theta$ results in a new vector $(y_1, y_2)$, where the components are calculated as:

$y_1 = x_1 \cos(i\theta) - x_2 \sin(i\theta)$ $y_2 = x_1 \sin(i\theta) + x_2 \cos(i\theta)$

Based on this structure, which statement best analyzes how the output vector's components are formed?

A positional encoding method transforms a two-dimensional vector $(x_1, x_2)$ at position $i$ into a new vector $(y_1, y_2)$ using the equations:

$y_1 = x_1 \cos(i\theta) - x_2 \sin(i\theta)$ $y_2 = x_1 \sin(i\theta) + x_2 \cos(i\theta)$

This transformation is considered non-linear with respect to the input vector $(x_1, x_2)$ because it involves trigonometric functions.

Matrix Representation of a 2D Rotary Transformation