Learn Before
When encoding sequential data by applying a rotational transformation for each step, the final orientation of a vector for the third item in a sequence is fundamentally different and cannot be achieved by a single rotational operation from the initial state.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Comprehension in Revised Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formula for Applying RoPE Rotation 't' Times
A system encodes the position of items in a sequence by applying a series of rotational transformations to an initial vector. The vector for the item at position 'm' is obtained by applying 'm' successive rotations of a fixed angle. Given that any number of successive rotations can be mathematically combined into a single, equivalent rotation, what is the primary advantage of this property for the system?
Equivalent Rotational Transformation
When encoding sequential data by applying a rotational transformation for each step, the final orientation of a vector for the third item in a sequence is fundamentally different and cannot be achieved by a single rotational operation from the initial state.