Learn Before
Model-Assisted Safety and Rule-Based Reward Models - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Rule-Based Reward Models (RBRMs)
Safety Metrics and Refusal Behavior - Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Safety Alignment and Rule-Based Reward Models - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Inputs and Rubric Classification Mechanism of RBRMs
A rule-based reward model (RBRM) processes three inputs: an optional prompt, the response output generated by the policy model, and a human-written rubric structured with explicit evaluation rules (often in multiple-choice format). The classifier evaluates the response against the rubric categories—such as a refusal in the desired style, a refusal in an undesired style (e.g., rambling or evasive), disallowed content, or a safe non-refusal—allowing the system to assign tailored rewards or penalties based on the classification outcome.
0
1
Tags
Prep Sessions
Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Ch.3 Model Alignment and Safety - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Model-Assisted Safety and Rule-Based Reward Models - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Ch.2 Post-Training Analysis and Safety - Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Safety Metrics and Refusal Behavior - Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Ch.2 Post-Training Alignment and Calibration - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Safety Alignment and Rule-Based Reward Models - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Related
Model-Assisted Safety Pipeline
Rule-Based Reward Models (RBRMs)
Inputs and Rubric Classification Mechanism of RBRMs
Function and Inputs of the RLHF Reward Model
Rule-Based Reward Models for Reasoning
How are rule-based reward models (RBRMs) categorized in terms of classifier type?
Rule-based reward models (RBRMs) rely solely on scalar human preference models to enforce safety policies.
During which training phase do rule-based reward models (RBRMs) supply additional reward signals to the policy model?
Explain the two key safety objectives that rule-based reward models (RBRMs) are designed to balance regarding model refusals.
Inputs and Rubric Classification Mechanism of RBRMs
Model-Assisted Safety Pipeline
Inputs and Rubric Classification Mechanism of RBRMs
Safety Metric Improvements in GPT-4
Model-Assisted Safety Pipeline
Rule-Based Reward Models (RBRMs)
Inputs and Rubric Classification Mechanism of RBRMs
Learn After
Order the steps involved in evaluating a policy response using a rule-based reward model (RBRM).
Identify the rubric category that the RBRM classifier should assign to this output, and explain what downstream action this classification enables the system to take.
Match each input to a rule-based reward model (RBRM) with its corresponding description.