Learn Before
Score-based Pruning of Prompts
A simple method for pruning a candidate pool of prompts is to use their evaluation scores. This technique involves retaining only a certain percentage of the top-performing prompts and discarding all others.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Related
A team is developing a system to find the best instructions for a language model on a complex task. They have generated 50,000 different candidate instructions. The full evaluation process for each candidate is computationally intensive and expensive. An engineer proposes implementing a preliminary, less accurate, but very fast filtering step to discard 95% of the candidates before the full, expensive evaluation begins. Which of the following statements best evaluates the primary trade-off of th
Resource-Constrained Prompt Optimization
Justifying the Pruning Step in Prompt Optimization
Learn After
Prompt Candidate Selection for a Chatbot
An AI development team is refining prompts for a new summarization tool. They have evaluated an initial pool of 10 candidate prompts, assigning each a performance score. To focus their efforts, they decide to prune the pool by retaining only the top 30% of prompts based on these scores. Given the scores below, which group of prompts will be selected for the next stage of optimization?
Scores: Prompt A (92), Prompt B (89), Prompt C (85), Prompt D (78), Prompt E (77), Prompt F (71), Prompt G (65)
An AI development team uses a simple method to refine their prompt candidates: they evaluate all prompts on a small dataset, assign each a performance score, and then discard the bottom 80%, keeping only the top 20% for further testing. What is the most significant risk of relying solely on this approach?