Training Objective as Maximization of the Performance Function

Derivation of the Policy Gradient Objective Function

Off-Policy Objective Function with Importance Sampling

An agent is operating under a policy parameterized by $\theta$. This policy can result in one of two possible trajectories. Trajectory A has a total reward of 20 and a 70% probability of occurring. Trajectory B has a total reward of -10 and a 30% probability of occurring. Given that the performance of a policy is measured by the expected cumulative reward over all possible trajectories ($J(\theta) = \sum_{\tau} \text{Pr}_{\theta}(\tau)R(\tau)$), what is the value of the performance function $J(\theta)$ for this policy?

Critique of the Expected Reward Objective

On-Policy Objective Function (Performance Measure)

Policy Performance Comparison