A new long-context language model, 'ContextCraft,' achieves a near-perfect score on a benchmark test that requires finding a single, specific fact hidden within a 200-page document. However, when deployed to a group of paralegals for beta testing, the feedback is overwhelmingly negative, with users reporting that the model's summaries of legal contracts are often incoherent and miss key clauses. Which statement best analyzes this situation?
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A new long-context language model, 'ContextCraft,' achieves a near-perfect score on a benchmark test that requires finding a single, specific fact hidden within a 200-page document. However, when deployed to a group of paralegals for beta testing, the feedback is overwhelmingly negative, with users reporting that the model's summaries of legal contracts are often incoherent and miss key clauses. Which statement best analyzes this situation?
Benchmark Performance vs. User Satisfaction
Designing a User-Centric Evaluation for a Customer Support AI