Learn Before
In a system that uses rotational transformations to encode positional information, where the rotation frequencies are defined as an exponential function of their dimension index, the frequencies of rotation are designed to decrease for higher-indexed pairs of dimensions.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Comprehension in Revised Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
In a system that uses rotational transformations to encode positional information, the frequencies of rotation for different pairs of dimensions (θ_k) are defined as an exponential function of their index (k). What is the primary analytical consequence of this design choice for the resulting positional encodings?
Evaluating Positional Encoding Schemes
Formula for RoPE Frequency Parameters (θ_k)
In a system that uses rotational transformations to encode positional information, where the rotation frequencies are defined as an exponential function of their dimension index, the frequencies of rotation are designed to decrease for higher-indexed pairs of dimensions.