Learn Before
You are tasked with improving a language model's ability to generate helpful and harmless responses. You decide to use a method that involves generating multiple potential responses to a prompt, scoring them with a separate quality-assessment model, and then using only the best-scoring responses to further train the original model. Arrange the following steps of this process in the correct logical order.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Comprehension in Revised Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Comparison of Rejection Sampling and RLHF
Adoption of Rejection Sampling in LLMs
Analyzing a Flawed Model Improvement Pipeline
You are tasked with improving a language model's ability to generate helpful and harmless responses. You decide to use a method that involves generating multiple potential responses to a prompt, scoring them with a separate quality-assessment model, and then using only the best-scoring responses to further train the original model. Arrange the following steps of this process in the correct logical order.
A machine learning team wants to improve a base language model's ability to follow instructions. They have already trained a separate, reliable 'reward model' that can score the quality of any given response. The team wants to use this reward model to enhance the base model's performance directly through a data-centric approach, avoiding more complex training paradigms. Which of the following strategies correctly describes the most effective and direct way to use the reward model for this purpose?