Formula for Soft Prompt Optimization by Minimizing KL Divergence

Derivation of the KL Divergence Objective for Policy Optimization

A machine learning model produces a probability distribution Q over a set of outcomes, aiming to approximate a true data distribution P. During evaluation, you observe that the divergence measure $D(P \|\| Q) = \sum_{x} P(x) \log\left(\frac{P(x)}{Q(x)}\right)$ is low, while the reverse measure $D(Q \|\| P) = \sum_{x} Q(x) \log\left(\frac{Q(x)}{P(x)}\right)$ is high. Based on these results, what is the most likely characteristic of the model's distribution Q?

Calculating Divergence Between Distributions

Choosing a Loss Function for Model Distillation