A reinforcement learning agent, in a state s_t with an estimated value V(s_t) = 50, takes an action. This action yields an immediate reward r = 5 and transitions the agent to a new state s_{t+1} with an estimated value V(s_{t+1}) = 40. Assuming a discount factor γ = 0.9, the agent's learning algorithm uses the quantity r + γV(s_{t+1}) - V(s_t) to update its policy. How should the agent interpret the outcome of this action?

Explaining Accelerated Learning in Reinforcement Learning

Equivalence of Advantage Estimation and Reward Shaping

In reinforcement learning, using the one-step advantage estimate, calculated as r + γV(s_{t+1}) - V(s_t), to update an agent's policy is a fundamentally distinct approach from training the agent with a shaped reward signal.