Evaluating an AI Preference Labeler
A company is using an AI to label pairs of potential customer service responses to create training data. Analyze the following scenario and the AI labeler's decision. Do you agree with the AI's preference? Justify your evaluation by explaining which response is superior for training a high-quality customer service model and why.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Criteria for a Good Customer Service Response
A company is using an AI to generate preference data to train a customer service chatbot. For the customer query, 'My order #ABC-123 was supposed to arrive yesterday but the tracking hasn't updated,' the system generates two possible responses. To create the most effective training data, which response should the AI be prompted to label as 'preferred,' and why?
Evaluating an AI Preference Labeler
You are tasked with creating a single piece of preference data to help train a customer service AI. Arrange the following steps in the correct logical order to accomplish this.