Formula for Soft Prompt Optimization via Log-Likelihood Maximization
The optimal soft prompt, denoted as , can be found by maximizing the log-probability of the target prediction (derived from the full context). This optimization is conditioned on the soft prompt and the original input . The formula is expressed as: This approach frames the optimization problem as a maximum likelihood estimation task, where the goal is to find the prompt that makes the desired output most probable.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Formula for Optimizing Soft Prompts via Context Compression
Formula for Soft Prompt Optimization via Log-Likelihood Maximization
Formula for Soft Prompt Optimization by Minimizing KL Divergence
An inference engine using a continuous batching strategy is currently processing a set of text generation requests that fully utilizes its processing capacity. At this point, a new, additional request arrives. What is the most likely immediate action the system's scheduler will take regarding this new request?
A language model is provided with a context
c('Translate the following sentence for a medical professional') and an inputz('Le patient présente une pyrexie'). The model computes the conditional probabilities for several potential English translations (y). Based on the principle of selecting the output that maximizes the conditional probability given the full context and input, which translation should the model choose as its prediction?Analyzing Contextual Influence on LLM Predictions
Formula for Soft Prompt Optimization via Log-Likelihood Maximization
Formula for Soft Prompt Optimization by Minimizing KL Divergence
A team is creating a soft prompt to summarize a complex user manual for a question-answering model. Their main objective is not just to get the single correct answer, but to ensure the model's uncertainty and its ranking of other plausible-but-incorrect answers are the same with the soft prompt as they were with the full manual. Which of the following optimization strategies best aligns with this specific objective?
Choosing an Optimization Strategy for Soft Prompts
A researcher is optimizing a soft prompt. With the original, long context, the model predicts the correct answer with 60% probability and a plausible alternative with 30% probability. The researcher's goal is to create a soft prompt that causes the model to predict the correct answer with over 95% probability, even if this significantly changes the probability of the alternative answer. Which optimization approach is better suited for this specific goal?
Learn After
A research team is developing a system to answer questions based on a large document. Instead of feeding the entire document into a language model for every question, they want to learn a compressed, continuous representation of the document (a 'soft prompt', σ). Their process is as follows:
- First, for a given question (z), they run the model with the full document to get a high-quality, 'gold standard' answer (ŷ).
- Next, they try to find the optimal soft prompt (σ) that, when paired with the original question (z), causes the model to produce that same 'gold standard' answer (ŷ).
They define the 'optimal' soft prompt as the one that makes the probability of generating the 'gold standard' answer as high as possible. Based on this optimization strategy, which statement best describes the primary goal?
Interpreting the Soft Prompt Optimization Formula
A team is training a soft prompt (σ) to help a language model generate a specific, high-quality target sentence (ŷ) when given an input (z). They are considering two different optimization objectives:
- Objective 1: Adjust the soft prompt σ to maximize the probability of the model generating the exact target sentence ŷ.
- Objective 2: Adjust the soft prompt σ so that the model's entire probability distribution over the next possible word matches the distribution it would have had if it were conditioned on the full, original context instead of the prompt.
Which statement best evaluates the fundamental difference in what these two objectives are trying to achieve?