Learn Before
Vector Reinterpretation for Rotational Transformation
A 4-dimensional token embedding is represented by the vector x = [1.0, 2.0, -3.0, 0.5]. To prepare this vector for a rotational transformation that encodes positional information, it is reinterpreted as a vector of complex numbers by grouping consecutive pairs. What are the resulting complex numbers?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Definition of RoPE-Encoded Token Embedding Notation (ei)
A 6-dimensional token embedding vector, represented as
v = [v1, v2, v3, v4, v5, v6], is being prepared for a rotational transformation to encode its position. Which of the following correctly describes how this vector is reinterpreted for the transformation process?Vector Reinterpretation for Rotational Transformation
To prepare a 4-dimensional token embedding vector
v = [1.0, 2.5, -0.5, 4.0]for a rotational transformation, it is first reinterpreted as a 2-dimensional vector of complex numbers. The first complex number in this new vector is1.0 + 2.5i. The second complex number is ____.When applying a rotational transformation to a 128-dimensional token embedding, the transformation applied to the complex number formed by the 1st and 2nd elements is dependent on the transformation applied to the complex number formed by the 3rd and 4th elements.
Formula for Multi-dimensional RoPE in Complex Space