Modeling Relative Position Offset via RoPE's Inner Product

The inner product of two token embeddings, x and y, at positions t and s respectively, is calculated after a rotational transformation using the formula: ⟨C(x, tθ), C(y, sθ)⟩ = (x'ȳ')e^(i(t-s)θ). In this formula, x' and ȳ' are complex number representations of the original embeddings. If both tokens are shifted by a constant amount k to new positions t+k and s+k, how does the inner product change?

Deconstructing the RoPE Inner Product Formula

The formula for the inner product of two RoPE-encoded tokens is given by ⟨C(x, tθ), C(y, sθ)⟩ = (x'ȳ')e^(i(t-s)θ). Match each component of this formula to its correct description, analyzing its specific role in the overall calculation.