Formula

RoPE 2D Vector Rotation Formula

The Rotary Positional Embedding (RoPE) for a 2-dimensional vector x=[x1,x2]\mathbf{x} = [x_1, x_2] is defined as a rotation by an angle θ\theta. This is achieved by multiplying the vector with a 2x2 rotation matrix RθR_\theta. The operation and its result are given by:

Ro(x,θ)=xRθ=[x1x2][cosθsinθsinθcosθ]=[cosθx1sinθx2sinθx1+cosθx2]\begin{aligned} \mathrm{Ro}(\mathbf{x}, \theta) &= \mathbf{x}R_\theta \\ &= \begin{bmatrix} x_1 & x_2 \end{bmatrix} \begin{bmatrix} \cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{bmatrix} \\ &= \begin{bmatrix} \cos \theta \cdot x_1 - \sin \theta \cdot x_2 & \sin \theta \cdot x_1 + \cos \theta \cdot x_2 \end{bmatrix} \end{aligned}
Image 0

0

1

Updated 2026-04-29

Contributors are:

Who are from:

Tags

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences