Learn Before
A researcher defines a new rotary position embedding function, Ro_new, for a token x_i at position i. The new function is defined as Ro_new(x_i, iθ) = Ro(x_i, (i+c)θ), where Ro is the original function and c is a constant offset. According to the general equivalence principle, this can be written as Ro_new(x_i, iθ) = Ro(x_i, iθ'). What is the correct expression for the transformed position parameter iθ'?
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formula for RoPE with Linear Positional Interpolation
A researcher defines a new rotary position embedding function,
Ro_new, for a tokenx_iat positioni. The new function is defined asRo_new(x_i, iθ) = Ro(x_i, (i+c)θ), whereRois the original function andcis a constant offset. According to the general equivalence principle, this can be written asRo_new(x_i, iθ) = Ro(x_i, iθ'). What is the correct expression for the transformed position parameteriθ'?RoPE Scaling Transformation Equivalence
Equivalence of RoPE Modification Strategies
Analysis of a Flawed RoPE Modification