Learn Before
Evaluating Prompt Demonstrations
A developer wants to use a language model to classify user feedback into two categories: 'Bug Report' or 'Feature Request'. They have created two potential sets of example input-output pairs to include in the prompt.
Set A:
- Input: 'The app crashes when I click the save button.' -> Output: 'Bug Report'
- Input: 'It would be great if you could add a dark mode.' -> Output: 'Feature Request'
Set B:
- Input: 'The login button is broken and doesn't respond.' -> Output: 'Bug Report'
- Input: 'The export-to-PDF feature is not working correctly.' -> Output: 'Bug Report'
Which set of examples is more effective for teaching the model this specific classification task, and why? Justify your reasoning.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A developer is trying to get a language model to extract product codes from customer emails. They provide the following examples in the prompt before asking the model to process a new email:
Example 1: Input: 'Hi, my SuperWidget model SW-1000 is broken.' Output: 'SW-1000'
Example 2: Input: 'I need a replacement part for my SuperWidget Pro, model number SW-2500.' Output: 'SW-2500'
New Email: Input: 'My GigaGadget GG-500 won't turn on.'
The model incorrectly outputs 'SW-500'. Based on an analysis of the provided examples, what is the most likely reason for this error?
Evaluating Prompt Demonstrations
Evaluating and Improving Prompt Demonstrations
Learning Output Formatting from Demonstrations