Learn Before
A startup is developing a language model to provide personalized financial advice to a global audience. To ensure the model's advice is safe and helpful, they plan to fine-tune it using preference data collected from a small team of 10 financial analysts, all from the company's headquarters in New York City. Based on the known challenges of using human-provided data for model alignment, what is the most critical potential flaw in this strategy?
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
AI Feedback as a Solution to Human Feedback Limitations
A startup is developing a language model to provide personalized financial advice to a global audience. To ensure the model's advice is safe and helpful, they plan to fine-tune it using preference data collected from a small team of 10 financial analysts, all from the company's headquarters in New York City. Based on the known challenges of using human-provided data for model alignment, what is the most critical potential flaw in this strategy?
Analyzing Alignment Challenges in a Global Chatbot Project
Evaluating a Claim of Perfect Model Alignment