Solution for RoPE Base Scaling Factor (λ)

An engineer is adapting a language model to handle longer text sequences. The goal is to find a scaling factor, λ, for the positional encoding base, b. The method involves setting the period of the highest frequency component in the new, adapted model equal to the period of a model scaled by linear interpolation. The dimensionality of the embeddings is d, the original sequence length is m_l, and the new sequence length is m. This constraint is captured by the following equation: $2\pi \cdot (\lambda b)^{\frac{2(\frac{d}{2}-1)}{d}} = \frac{m}{m_l} \cdot 2\pi \cdot b^{\frac{2(\frac{d}{2}-1)}{d}}$ Which part of this equation represents the period of the highest frequency dimension for the new model being developed?

An engineer is adapting a language model to process sequences twice as long as its original design (i.e., m = 2 * m_l). They use a method where the period of the highest frequency component in the new model is set equal to that of a linearly scaled model. This relationship is captured by the equation: $2\pi \cdot (\lambda b)^{\frac{d-2}{d}} = \frac{m}{m_l} \cdot 2\pi \cdot b^{\frac{d-2}{d}}$ Given that the embedding dimensionality d is greater than 2 and the original base b is a positive constant, how must the scaling factor λ change to satisfy this constraint for the new, longer sequence length?

Evaluating a Proposed Simplification