Learn Before
Evaluating LLM vs. Fine-Tuned Models for Classification
A financial tech company needs to build a system to classify news articles into categories like 'Mergers & Acquisitions', 'Earnings Reports', 'Executive Changes', and 'Market Rumors'. They are evaluating two approaches: (1) Fine-tuning a smaller, specialized classification model, and (2) Using a large, general-purpose language model through a third-party API. Critically evaluate the decision to use the large, general-purpose model. In your evaluation, identify and discuss at least three distinct, unresolved challenges or risks associated with this approach that are less prominent with the fine-tuned model.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Proposal for LLM-based Classification System
A financial services company is developing a system to classify customer support emails into one of three categories: 'Account Inquiry', 'Technical Issue', or 'Complaint'. They are considering two different prompting strategies for their Large Language Model.
- Strategy A: "Classify the following email into one of these categories: Account Inquiry, Technical Issue, Complaint. Email: [email text]"
- Strategy B: "Analyze the following email and determine if it is an Account Inquiry, a Technical Issue, or a Complaint. If it does not fit any of these, label it as 'Other'. Email: [email text]"
Which of the following statements best evaluates the potential risks associated with these strategies?
Challenge of Prompting LLMs for Many-Category Classification
Evaluating LLM vs. Fine-Tuned Models for Classification
A team is deploying a text classification system using a large language model. They encounter several unexpected behaviors. Match each observed behavior with the most likely underlying issue.