Learn Before
In a specific positional encoding method, pairs of embedding dimensions are rotated using a 2x2 matrix defined as: where t is the token's position and \theta_k is a frequency. Given a token at position t = 2 and a frequency \theta_k = \pi/4, which matrix correctly represents the rotation?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.3 Prompting - Foundations of Large Language Models
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Euclidean Representation of RoPE Rotation
In a specific positional encoding method, pairs of embedding dimensions are rotated using a 2x2 matrix defined as: where
tis the token's position and\theta_kis a frequency. Given a token at positiont = 2and a frequency\theta_k = \pi/4, which matrix correctly represents the rotation?In a certain positional encoding method, the rotation of a pair of dimensions is achieved using the matrix , where
tis the position and\thetais a frequency parameter.Applying a Rotational Transformation Matrix