Applying knowledge distillation to context compression involves treating the full-context prediction as the teacher model and the compressed-context prediction as the student model. Unlike standard context distillation where the compressed context uses discrete tokens, this method distills the context $\mathbf{c}$ into real-valued vectors $\sigma$, which act as prompt embeddings. Furthermore, the teacher and student models are not required to share the same architecture; typically, a stronger model serves as the teacher, while a smaller, more efficient model acts as the student.