Impact of the Similarity Function in Soft Prompt Optimization
A researcher is using the formula below to learn an optimal soft prompt, σ, that compresses a longer context, c. The goal is for the model's prediction with the soft prompt (ŷ_σ) to match its prediction with the full context (ŷ).
hat(σ) = argmin_σ s(ŷ, ŷ_σ)
The researcher is considering two different definitions for the similarity function s(·, ·):
- Function A: A simple mismatch penalty. The function returns 0 if the single most likely output token for
ŷis identical to the single most likely output token forŷ_σ. Otherwise, it returns 1. - Function B: A distributional divergence measure. This function calculates the difference between the entire probability distributions over all possible output tokens produced with the full context and the soft prompt, respectively.
Analyze the likely difference in the behavior of the resulting soft prompt (hat(σ)) when optimized using Function A versus Function B, particularly for tasks that require generating multi-word, coherent answers.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A machine learning engineer is training a soft prompt, σ, to replace a lengthy context, c. They use the following optimization formula, where s(·,·) is a function measuring the difference between two predictions:
hat(σ) = argmin_σ s(hat(y), hat(y)_σ)Here,
hat(y)is the model's prediction with the full context c, andhat(y)_σis the prediction with the soft prompt σ. After training, the engineer observes that for many inputs, the value ofs(hat(y), hat(y)_σ)is consistently high. What does this observation most directly imply about the outcome of the training process?Impact of the Similarity Function in Soft Prompt Optimization
In the context of learning a compressed representation of a long text, consider the optimization formula:
hat(σ) = argmin_σ s(hat(y), hat(y)_σ), wherehat(y)is the prediction from the full text andhat(y)_σis the prediction from the compressed representationσ. If the functions(·,·)were changed from a dissimilarity measure (e.g., a loss function) to a similarity measure (e.g., a cosine similarity score), theargminoperator should be replaced withargmaxto correctly identify the optimal compressed representationhat(σ).