Learn Before
Independent Rotational Transformation in RoPE
A 6-dimensional embedding vector is transformed by post-multiplication with a 6x6 block-diagonal rotation matrix. This matrix is composed of three 2x2 rotation blocks along its diagonal, with all other elements being zero. Analyze the following scenario and explain your reasoning.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.2 Generative Models - Foundations of Large Language Models
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Consider a 4-dimensional row vector . This vector undergoes a rotational transformation by being post-multiplied by a 4x4 block-diagonal matrix. This matrix consists of two 2x2 rotation blocks along its diagonal. The first block rotates the first pair of vector components by an angle of , and the second block rotates the second pair of components by an angle of . Given that each 2x2 rotation block for an angle is defined as , what is the resulting vector?
Consider the transformation of a d-dimensional vector by post-multiplication with a d-dimensional block-diagonal rotation matrix, as used in Rotary Positional Embeddings. This transformation will alter the Euclidean norm (magnitude) of the vector.
Independent Rotational Transformation in RoPE