1Cademy - Decoupled Reward and Efficiency Formulations in Evo-GDPO

Learn Before

Stage III: Evolutionary Group-Decoupled Policy Optimization - Dynamic Agent Scaffolding: Synthesis, Diagnostic Repair, and Evolutionary Optimization @ University of Michigan - Ann Arbor
Evolutionary Group Decoupled Policy Optimization (Evo-GDPO)

Formula

Decoupled Reward and Efficiency Formulations in Evo-GDPO

In Evo-GDPO, candidate harnesses are evaluated against an archive incumbent with baseline statistics $(b_r, b_\ell, b_\kappa)$ . Rather than combining raw performance and resource metrics into a scalar reward, Evo-GDPO decouples evaluation into three distinct unnormalized channels:

$R_i^{\text{rew}} = r_i + \lambda_{\text{evo}} [r_i - b_r]_+$ $R_i^{\text{lat}} = \mathbb{I}[r_i \ge b_r] [b_\ell - \bar{\ell}_i]_+$ $R_i^{\text{cost}} = \mathbb{I}[r_i \ge b_r] [b_\kappa - \bar{\kappa}_i]_+$

where $\lambda_{\text{evo}} \ge 0$ controls the evolutionary bonus and $\mathbb{I}[\cdot]$ is an indicator function. The reward signal $R_i^{\text{rew}}$ serves as the primary objective, whereas efficiency advantages for latency $R_i^{\text{lat}}$ and monetary cost $R_i^{\text{cost}}$ activate exclusively when the candidate preserves or exceeds the incumbent's task reward ( $r_i \ge b_r$ ).

0

1

Updated 2026-10-02

Contributors are:

Who are from:

University of Michigan - Ann Arbor

🏆 1

References

Learn Before

Related

Learn After