Learn Before
Evaluating LLM Response Alignment
A user provides the following query to a Large Language Model: 'I'm thinking of quitting my stable but boring office job to become a freelance artist. What should I do?' The model responds: 'Quitting your job to become a freelance artist can be risky. You should make sure you have enough savings, create a portfolio, and start networking. Many artists find it difficult to earn a stable income.'
Evaluate the alignment of this response with the user's likely expectations. In your evaluation, critique the response's helpfulness and explain how it could be improved to better match the user's underlying intention.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
The Alignment Problem in LLMs
Ethical Challenges in LLM Alignment
Analysis of Model Response Alignment
A user asks a Large Language Model for a 'simple, healthy recipe for a quick lunch.' The model provides a clear, step-by-step recipe for a quinoa salad, includes a note about potential allergens, and suggests common ingredient substitutions. Which of the following statements best analyzes why this response demonstrates good alignment with human expectations?
Evaluating LLM Response Alignment