Learn Before
Pruning the Prompt Candidate Pool
If the prompt candidate pool contains a large number of prompts, a standard pruning step is employed to discard unpromising options. Filtering out these less effective prompts significantly reduces the computational burden in the subsequent evaluation and expansion steps of the optimization process.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Evaluating Prompts with Pre-defined Metrics
Using Log-Likelihood to Evaluate Prompts
Pruning the Prompt Candidate Pool
A team is developing a system to classify customer feedback emails as 'Urgent' or 'Not Urgent'. They have created a set of 20 different instruction prompts to guide a language model in this classification task. To determine the best prompt, they select one sample 'Urgent' email and test each of the 20 prompts on it. They decide to choose the prompt that successfully leads the model to classify this single email as 'Urgent'. What is the most significant flaw in this evaluation strategy?
A developer has created a set of candidate prompts to make a language model summarize news articles. To find the best prompt, each one must be evaluated. Arrange the following actions into the correct logical sequence for evaluating a single candidate prompt across a dataset of articles.
Evaluating Prompts for a Customer Support Chatbot
Learn After
Score-based Pruning of Prompts
A team is developing a system to find the best instructions for a language model on a complex task. They have generated 50,000 different candidate instructions. The full evaluation process for each candidate is computationally intensive and expensive. An engineer proposes implementing a preliminary, less accurate, but very fast filtering step to discard 95% of the candidates before the full, expensive evaluation begins. Which of the following statements best evaluates the primary trade-off of this proposal?
Resource-Constrained Prompt Optimization
Justifying the Pruning Step in Prompt Optimization