Worth Function in Plackett-Luce for RLHF Reward Modeling

A team is training a reward model using human feedback. Instead of collecting simple pairwise comparisons (e.g., 'Response A is better than Response B'), they have collected full rankings of four responses for each prompt. They decide to use a listwise ranking model to train their reward model on this data. What is the primary conceptual advantage of this listwise approach compared to an alternative approach of simply breaking each ranked list down into all possible pairs and aggregating their individual losses?

Reward Model Training Strategy

Reward Model's Role in Listwise Preference Learning