Comparison of Rejection Sampling and RLHF

Adoption of Rejection Sampling in LLMs

Analyzing a Flawed Model Improvement Pipeline

You are tasked with improving a language model's ability to generate helpful and harmless responses. You decide to use a method that involves generating multiple potential responses to a prompt, scoring them with a separate quality-assessment model, and then using only the best-scoring responses to further train the original model. Arrange the following steps of this process in the correct logical order.

A machine learning team wants to improve a base language model's ability to follow instructions. They have already trained a separate, reliable 'reward model' that can score the quality of any given response. The team wants to use this reward model to enhance the base model's performance directly through a data-centric approach, avoiding more complex training paradigms. Which of the following strategies correctly describes the most effective and direct way to use the reward model for this purpose?