Policy Gradient Utility for Sequence Generation

A research team is training a language model to generate helpful and harmless dialogue responses. They define a utility function for a given input x and a generated response y as: U(x, y) = (0.8 * Helpfulness_Score) - (0.2 * Harmfulness_Score). The team's objective is to find the model parameters, θ, that maximize the average utility across a large dataset of interactions. Which of the following loss functions, L(θ), should the team minimize to achieve this objective?

A machine learning model is being trained with the objective of maximizing a specific utility function, U(x, y; θ), which measures the quality of its outputs. The loss function used for training is defined as L(θ) = E[(x,y)~D][U(x, y; θ)]. True or False: Minimizing this loss function L(θ) will successfully train the model to achieve its objective.

Diagnosing a Flawed Training Objective