In a policy optimization algorithm, what is the primary analytical advantage of constructing the local approximation for an update step around the specific point where the current policy's parameters are identical to the reference policy's parameters?

PPO Objective at the Reference Point

In the context of Proximal Policy Optimization, the gradient of the objective function is zero at the specific point where the current policy parameters equal the reference policy parameters, signifying that no further update is necessary from this point.