Formula

Euclidean Representation of RoPE Transformation

The Rotary Position Embedding (RoPE) transformation can be represented in d-dimensional Euclidean space using the formula: Ro(x,tθ)=[x1x2xd][Rtθ1Rtθ2Rtθd/2]\mathrm{Ro}(\mathbf{x}, t\theta) = \begin{bmatrix} x_1 & x_2 & \dots & x_d \end{bmatrix} \begin{bmatrix} R_{t\theta_1} & & & \\ & R_{t\theta_2} & & \\ & & \ddots & \\ & & & R_{t\theta_{d/2}} \end{bmatrix} In this equation, a d-dimensional vector x\mathbf{x} is transformed by multiplying it with a block-diagonal matrix. This matrix consists of d/2d/2 individual 2x2 rotation matrices, RtθkR_{t\theta_k}, along its diagonal. This structure effectively pairs up the components of x\mathbf{x} (e.g., x1x_1 with x2x_2, x3x_3 with x4x_4, etc.) and applies a separate 2D rotation to each pair.

Image 0

0

1

Updated 2025-10-08

Contributors are:

Who are from:

Tags

Ch.3 Prompting - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences