Learn Before
Permuted Input Vector in RoPE
In the element-wise formulation of Rotary Positional Embeddings (RoPE), a permuted version of the input vector is required. This vector is constructed by swapping each pair of adjacent elements and negating the first element of the swapped pair, resulting in . For a full d-dimensional vector , the permuted vector is given by: This vector is a key component in the calculation, where it is multiplied element-wise with the sine vector.

0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Permuted Input Vector in RoPE
A transformation is applied to a d-dimensional vector at position using a set of frequencies . The transformation is defined by the following element-wise formula: where denotes element-wise multiplication.
Given the input vector , position , and frequencies , , what is the resulting vector ?
Analyzing Components of a Vector Transformation
Consider the element-wise formula for applying a rotational transformation to a d-dimensional vector . The formula involves summing two vectors derived from element-wise products.
True or False: The first vector in the sum is the product of the original input vector and a sine vector, and the second vector is the product of a permuted input vector (e.g., ) and a cosine vector.
Learn After
A specific vector transformation operates on a d-dimensional vector by processing it in pairs. For each adjacent pair of elements
(v_{2k-1}, v_{2k}), the elements are swapped, and then the first element of the new pair is negated. Given an input vectorx = [10, 20, 30, 40], what is the resulting vector after this transformation is applied?Reversing a Vector Permutation
Iterative Vector Transformation