Mitigating Bias in Automated Preference Data Generation
A research team is using a single, powerful language model to automate the creation of a preference dataset. The model first generates several responses to a prompt, and then the same model evaluates and ranks these responses. Describe a significant potential issue with this approach regarding the quality of the final dataset, and propose one specific strategy the team could implement to address this issue.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Techniques for Generating Diverse Outputs in RLHF
A development team is creating a large preference dataset. They use a single, highly advanced language model for the entire process: for each input, the model generates two distinct responses, and then the same model is prompted again to choose which of the two responses is better. What is the most significant risk to the quality and utility of the final dataset produced by this method?
Evaluating a Data Generation Strategy
Mitigating Bias in Automated Preference Data Generation