Composition Property of Rotations
The composition property of rotations states that performing multiple successive rotations on a vector is equivalent to applying a single, combined rotation. This principle is based on the mathematical fact that the result of composing several rotations is always another rotation. Consequently, rotating a vector 't' times by an angle θ is a valid operation that results in a single rotation by a cumulative angle.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Composition Property of Rotations
Representing 2D Vector Rotation in Complex Space
Definition of the 2D Rotation Matrix
Consider a system where the position of a token in a sequence is encoded by rotating its initial vector embedding,
x. The total angle of rotation is directly proportional to the token's position,m. If the vector for a token at position 3 is obtained by rotatingxby a total angle of3θ, what is the correct transformation to find the vector for the same token at position 9?A positional encoding system represents a token's position by sequentially rotating its initial vector embedding, denoted as
x, by a fixed angleθfor each step forward in a sequence. Arrange the following vector states to show the correct order of transformations for a token as its position advances from 1 to 3.In a system that encodes sequential position, an initial vector
xis transformed to represent positionmby applying a cumulative rotation, resulting in vectorv_m. Similarly, the vector for positionm+1isv_{m+1}. Based on this mechanism, what is the direct geometric transformation that relatesv_mtov_{m+1}?
Learn After
Formula for Applying RoPE Rotation 't' Times
A system encodes the position of items in a sequence by applying a series of rotational transformations to an initial vector. The vector for the item at position 'm' is obtained by applying 'm' successive rotations of a fixed angle. Given that any number of successive rotations can be mathematically combined into a single, equivalent rotation, what is the primary advantage of this property for the system?
Equivalent Rotational Transformation
When encoding sequential data by applying a rotational transformation for each step, the final orientation of a vector for the third item in a sequence is fundamentally different and cannot be achieved by a single rotational operation from the initial state.