Learn Before
Challenges in Optimizing Prompt Instructions
Learning optimal instructions is particularly difficult due to two main factors. First, pre-trained LLMs are often not adept at predicting an instruction's quality. Second, the process of evaluating instructions on downstream tasks is computationally intensive, which makes optimization costly and hinders broad exploration of different instruction styles.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Challenges in Optimizing Prompt Instructions
Comparison of Optimizing Prompt Instructions vs. Demonstrations
A team is developing an automated system to improve a Large Language Model's performance on a text classification task. The goal is to make the model more accurate at identifying the primary topic of a given document. Which of the following strategies best represents the specific process of optimizing the prompt's instructions?
Evaluating an Optimized Prompt Instruction for Summarization
Analyzing the Effectiveness of Prompt Instructions
Learn After
A research team is trying to find the most effective instruction to guide a large language model in summarizing legal documents. They first try asking the model to rate a list of 100 different candidate instructions on a scale of 1-10 for 'clarity and effectiveness'. They then discover that the model's ratings do not correlate well with which instructions actually produce the best summaries when tested. Furthermore, the process of generating and evaluating summaries for all 100 instructions is taking several days and consuming their entire computation budget. Which statement best analyzes the fundamental difficulties the team is facing?
Evaluating a Prompt Optimization Strategy
Critique of a Prompt Optimization Method
The most efficient method for optimizing a prompt's instruction for a new task is to have the large language model (LLM) score a list of potential instructions for quality, and then select the highest-scoring one for use, thereby avoiding the high computational cost of testing each instruction on the actual task.