Learn Before
Applying a Rotational Transformation Matrix
In a certain positional encoding method, a pair of embedding dimensions, represented by the row vector v = [x_1, x_2], is transformed by post-multiplying it with the following 2x2 rotation matrix: Where α represents the rotation angle. Determine the components of the resulting vector v' = vR. Express your answer for the two components, x'_1 and x'_2, in terms of x_1, x_2, cos α, and sin α.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.3 Prompting - Foundations of Large Language Models
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Euclidean Representation of RoPE Rotation
In a specific positional encoding method, pairs of embedding dimensions are rotated using a 2x2 matrix defined as: where
tis the token's position and\theta_kis a frequency. Given a token at positiont = 2and a frequency\theta_k = \pi/4, which matrix correctly represents the rotation?In a certain positional encoding method, the rotation of a pair of dimensions is achieved using the matrix , where
tis the position and\thetais a frequency parameter.Applying a Rotational Transformation Matrix