An agent is being trained using a policy gradient method. After each update to its decision-making process (the policy), the experiences (trajectories) it previously collected are no longer perfectly representative of its new behavior. This mismatch can lead to inaccurate estimates of the value of those past trajectories, causing instability in the training process. Which of the following approaches directly addresses this issue by adjusting the value calculation to account for the change in the policy?
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Policy Gradient Objective with Importance Sampling
An agent is being trained using a policy gradient method. After each update to its decision-making process (the policy), the experiences (trajectories) it previously collected are no longer perfectly representative of its new behavior. This mismatch can lead to inaccurate estimates of the value of those past trajectories, causing instability in the training process. Which of the following approaches directly addresses this issue by adjusting the value calculation to account for the change in the policy?
Evaluating Training Strategies for a Robotic Arm
Addressing Data Mismatch in Policy Gradient Training