Definition

Definition of RoPE-Encoded Token Embedding Notation (ei)

In Rotary Positional Embeddings (RoPE), the final embedding for a token xix_i at position ii is denoted as eie_i. This embedding is the result of applying the RoPE transformation, Ro\text{Ro}, to the original token embedding xix_i with the positional angle iθi\theta. The parameter set θ\theta is defined as θ={θ1,...,θd/2}\theta = \{\theta_1, ..., \theta_{d/2}\}. The formal definition of the embedding is: ei=Ro(xi,iθ)e_i = \text{Ro}(x_i, i\theta)

0

1

Updated 2025-10-08

Contributors are:

Who are from:

Tags

Ch.3 Prompting - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Learn After