Case Study

Evaluating a Prompt for a Custom Classification Task

A software development team wants to use a large language model to automatically categorize user feedback for their new, specialized 3D architectural modeling software. They need to sort the feedback into three specific, internally-defined categories: 'Bug Report', 'Feature Request', and 'Usability Issue'. They have drafted the following prompt to give to the model for each piece of feedback:

'Classify the following user feedback into one of these categories: Bug Report, Feature Request, or Usability Issue.'

Evaluate the effectiveness of this prompt for this specific task. Explain why it is likely to produce inaccurate or inconsistent results and what key information is missing.

0

1

Updated 2025-10-06

Contributors are:

Who are from:

Tags

Ch.3 Prompting - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Evaluation in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science