Learn Before
Model-Assisted Safety and Rule-Based Reward Models - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Safety Metrics and Refusal Behavior - Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Safety Alignment and Rule-Based Reward Models - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Model-Assisted Safety Pipeline
The model-assisted safety pipeline is an alignment framework that uses the language models themselves as tools to steer model behavior toward safe and nuanced responses. Designed to address failure modes in standard RLHF—such as generating unsafe advice or becoming overly cautious by hedging and refusing benign queries—the framework consists of two primary components: an expanded set of safety-relevant training prompts and rule-based reward models (RBRMs) that supply targeted feedback during reinforcement learning fine-tuning.
0
1
Tags
Prep Sessions
Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Ch.3 Model Alignment and Safety - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Model-Assisted Safety and Rule-Based Reward Models - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Ch.2 Post-Training Analysis and Safety - Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Safety Metrics and Refusal Behavior - Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Ch.2 Post-Training Alignment and Calibration - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Safety Alignment and Rule-Based Reward Models - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Related
Model-Assisted Safety Pipeline
Rule-Based Reward Models (RBRMs)
Inputs and Rubric Classification Mechanism of RBRMs
Function and Inputs of the RLHF Reward Model
Rule-Based Reward Models for Reasoning
Model-Assisted Safety Pipeline
Inputs and Rubric Classification Mechanism of RBRMs
Safety Metric Improvements in GPT-4
Model-Assisted Safety Pipeline
Rule-Based Reward Models (RBRMs)
Inputs and Rubric Classification Mechanism of RBRMs
Learn After
Which set of failure modes in standard reinforcement learning from human feedback (RLHF) is the model-assisted safety pipeline designed to address?
Rule-Based Reward Models (RBRMs)
Within this alignment framework, ___ themselves are used as tools to steer model behavior.
What specific role do rule-based reward models (RBRMs) play within the model-assisted safety pipeline?