Learn Before
Evaluating Core Difficulties in Model Behavior Guidance
Imagine you are tasked with creating a set of rules for a powerful AI assistant to ensure it is always 'helpful and harmless'. Critically evaluate why simply creating a comprehensive list of rules and training the AI on examples of good behavior is an insufficient strategy. In your answer, analyze at least two distinct underlying problems that make this task fundamentally difficult, connecting the nature of human expectations to the practical limitations of training such a system.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.4 Alignment - Foundations of Large Language Models
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Shift in LLM Alignment from Predefined Tasks to Real-World Interaction
Impracticality of Achieving Alignment Solely Through Pre-training
Need for Diverse Alignment Methods
Insufficiency of Data Fitting for Value Alignment
Difficulty of Encoding Human Values in Datasets
Inarticulacy of Human Preferences as an Alignment Challenge
Goodhart's Law
Real-World Complexity as an Alignment Challenge
Specification Gaming in AI Alignment
Alignment Challenges as a Motivator for AI Research
Diversity and Fluidity of Human Values as an Alignment Challenge
Analysis of an LLM Alignment Failure
A development team building a chatbot aims for it to be 'helpful' to all users. They discover that behaviors praised as helpful by users in one country are criticized as intrusive by users in another. This issue persists even after training the model on vast, culturally diverse datasets. Which fundamental challenge in guiding a model's behavior does this scenario best illustrate?
Evaluating Core Difficulties in Model Behavior Guidance
Challenge of Defining Human Values for AI Objectives