Decomposing a Ranked List into Pairwise Preferences
A common method for training a preference model is to take a ranked list of responses and break it down into all its constituent pairwise preferences. For example, the ranking A > B > C is decomposed into three preferences: A is preferred over B, A is preferred over C, and B is preferred over C. Following this method, if a human annotator provides a ranked list of 5 distinct responses, how many individual pairwise preferences will be extracted from this single list for the loss calculation?
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Listwise Loss Formula from Accumulated Pairwise Comparisons
A human annotator is given four model-generated responses (A, B, C, D) to a prompt and ranks them in order of preference from best to worst as: C > A > D > B. To train a preference model, a loss function is calculated by summing the individual losses for every pairwise comparison implied by this ranking. Which of the following sets represents all the pairwise preferences that would be used in this loss calculation?
Decomposing a Ranked List into Pairwise Preferences
Evaluating Preference Model Performance with Listwise Loss