Chatbot Performance Analysis
Based on the data generation strategy described in the case study, identify and explain the most probable cause for the chatbot's poor performance with real-world user queries.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Generating Inputs and Outputs for Comprehensive Fine-Tuning
Chatbot Performance Analysis
A development team is fine-tuning a large language model to act as a technical support chatbot. To create a large training dataset, they use a powerful base model to generate responses to a set of 10,000 technical questions curated by their internal support staff. After deployment, the chatbot excels at answering questions similar to those in the curated set but struggles significantly with novel or unusually phrased queries from real users. Which of the following best analyzes the primary weakness in their data generation strategy?
Evaluating Data Generation Strategies for Model Generalization