In the Stage I preference loss formulation $\mathcal{L}_{\text{I}}^{\text{pref}}(\theta)$, what is the function of the hyperparameter $\beta_{\text{pref}}$?

In the Stage I preference loss $\mathcal{L}_{\text{I}}^{\text{pref}}(\theta)$, the reference model $p_{\text{ref}}$ is actively updated alongside the policy parameters $\theta$.

What specific generation capability does the supervised fine-tuning loss $\mathcal{L}_{\text{I}}^{\text{gen}}(\theta)$ instill in the model?

Describe the two primary criteria that candidate scaffolds are biased toward by the preference loss $\mathcal{L}_{\text{I}}^{\text{pref}}(\theta)$, and explain the function of the weighting factor $\Delta_{\text{val}}$.

The training objective for Stage I in JIT-Agent couples reference-anchored preference optimization with supervised ___ learning into a combined loss.

Arrange the mathematical operations in the order they are evaluated within the preference loss expectation to compute the weighted argument of the loss.

Explain the structural impact of the configuration in Experiment A on the overall loss function, and describe the flaw introduced in Experiment B by fixing $\Delta_{\text{val}} = 1.0$.