Stabilizing Policy Gradient Learning in a High-Variance Environment

A reinforcement learning agent is being trained using a policy gradient method. During training, the agent's performance is highly erratic, and the estimated gradients for policy updates have very high variance. Which of the following changes to the gradient estimation process is most directly aimed at stabilizing learning by reducing this variance?

Rationale for Using the Advantage Function