Learn Before
Evaluating and Improving Prompt Demonstrations
A developer is using a large language model to classify customer support tickets. The goal is to categorize each ticket as 'Billing Inquiry', 'Technical Support', or 'General Question'. The model is performing poorly, frequently misclassifying the tickets. Analyze the example input-output pairs provided in the prompt below. Identify at least two significant flaws in these examples and explain how these flaws likely contribute to the model's poor performance. Finally, suggest specific improvements.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A developer is trying to get a language model to extract product codes from customer emails. They provide the following examples in the prompt before asking the model to process a new email:
Example 1: Input: 'Hi, my SuperWidget model SW-1000 is broken.' Output: 'SW-1000'
Example 2: Input: 'I need a replacement part for my SuperWidget Pro, model number SW-2500.' Output: 'SW-2500'
New Email: Input: 'My GigaGadget GG-500 won't turn on.'
The model incorrectly outputs 'SW-500'. Based on an analysis of the provided examples, what is the most likely reason for this error?
Evaluating Prompt Demonstrations
Evaluating and Improving Prompt Demonstrations
Learning Output Formatting from Demonstrations