Learn Before
An AI development team uses a simple method to refine their prompt candidates: they evaluate all prompts on a small dataset, assign each a performance score, and then discard the bottom 80%, keeping only the top 20% for further testing. What is the most significant risk of relying solely on this approach?
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Prompt Candidate Selection for a Chatbot
An AI development team is refining prompts for a new summarization tool. They have evaluated an initial pool of 10 candidate prompts, assigning each a performance score. To focus their efforts, they decide to prune the pool by retaining only the top 30% of prompts based on these scores. Given the scores below, which group of prompts will be selected for the next stage of optimization?
Scores: Prompt A (92), Prompt B (89), Prompt C (85), Prompt D (78), Prompt E (77), Prompt F (71), Prompt G (65), Prompt H (64), Prompt I (58), Prompt J (51)
An AI development team uses a simple method to refine their prompt candidates: they evaluate all prompts on a small dataset, assign each a performance score, and then discard the bottom 80%, keeping only the top 20% for further testing. What is the most significant risk of relying solely on this approach?