Learn Before
Checking Reward Quality in Reinforcement Learning
When an RL agent produces a trajectory that performs worse than an expert trajectory, compare the reward function's score on the expert behavior with its score on the agent's behavior. If the expert behavior receives the higher score, focus on improving the RL algorithm; if it does not, the reward function is likely the problem and should be revised.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Classifying development-set mistakes with optimization verification
Approximate Scoring Followed by Approximate Search
Checking Whether a Translation Error Comes from Search or Scoring
Checking Reward Quality in Reinforcement Learning
A useful human output for optimization verification
What does this optimization check suggest when the reference candidate scores higher than the algorithm’s output?
Using the Optimization Verification Test
Optimization Check Symbols
How to carry out an optimization verification check
Using the Optimization Verification Check
Debugging a Translation Decoder
When to Attribute an Error to Optimization
When Is an Optimization Check Useful?
Checking the Source of a Performance Problem
Optimization checks can isolate the source of a training problem
Learn After
Diagnosing an RL problem with verified rewards
When Reward Scores Favor the Wrong Outcome
Interpreting _____ in reward verification
Match the trajectories and reward comparison to their meanings.
Reward Comparison for a Reinforcement Learning Check
What It Means When the Human Trajectory Outperforms the Learned One
Autonomous Drone Landing Reward Diagnosis
Deciding the RL component to fix
What does it mean if a human plan scores lower than an automated plan on the reward model?
What Optimization Verification Checks