Evaluating a Prompt for a Custom Classification Task
A software development team wants to use a large language model to automatically categorize user feedback for their new, specialized 3D architectural modeling software. They need to sort the feedback into three specific, internally-defined categories: 'Bug Report', 'Feature Request', and 'Usability Issue'. They have drafted the following prompt to give to the model for each piece of feedback:
'Classify the following user feedback into one of these categories: Bug Report, Feature Request, or Usability Issue.'
Evaluate the effectiveness of this prompt for this specific task. Explain why it is likely to produce inaccurate or inconsistent results and what key information is missing.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Few-Shot Learning
A data scientist wants to use a large language model to categorize internal company documents into three newly-defined, specific categories: 'Alpha Project Brief', 'Beta Project Brief', and 'Gamma Project Brief'. The model has not been specifically trained on this internal classification system. Which of the following prompts is best designed to achieve the most accurate and consistent results for this task?
Improving a Prompt for a Novel Classification Task
Evaluating a Prompt for a Custom Classification Task