Learn Before
Comparison of Process and Outcome Reward Models
Process Reward Models (PRMs) differ from Outcome Reward Models (ORMs) in the granularity of their feedback. PRMs offer a detailed, fine-grained supervisory signal by evaluating every intermediate step of a reasoning process, whereas ORMs provide a coarser signal by assessing only the final outcome.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Comparison of Process and Outcome Reward Models
Data Collection Challenges for Process Reward Models
Evaluating a Feedback Strategy for an AI Tutor
An AI development team is training a model to solve complex, multi-step mathematical problems. Their primary goal is to ensure the model learns a logically sound reasoning process, rather than just arriving at the correct final answer through flawed logic. Which of the following training components would be most effective for providing the detailed, step-by-step guidance needed to achieve this goal?
A research team is developing a language model to generate high-quality, step-by-step solutions to physics problems. To ensure the model's reasoning is sound at each stage, they are training a separate verifier model that provides a reward for each step. Arrange the following actions into the correct chronological sequence for this training and feedback process.
Learn After
Diagnosing Flawed Reasoning in Language Models
A team is training a language model to act as a programming assistant that generates code. They observe that the model sometimes produces functionally correct code (the outcome is right) but uses inefficient, non-standard, or difficult-to-maintain methods (the process is poor). Which of the following feedback strategies would be most effective at specifically improving the quality of the reasoning process, rather than just the correctness of the final output?
A research team is developing a large language model for different tasks. Match each training objective with the most appropriate feedback strategy.
Comparing Outcome-Based and Process-Based Evaluations of Math Responses