Learn Before
Calculating Log-Probability for a Ranked List
A model is designed to rank three candidate responses {A, B, C} for a given prompt. The model determines the final ranking by making a sequence of choices. At each step, it computes the probability of selecting the best response from the set of options not yet ranked.
Given the following conditional probabilities for the sequence A > B > C:
- The probability of selecting A first from the set {A, B, C} is 0.5.
- The probability of selecting B second from the remaining set {B, C} is 0.7.
- The probability of selecting C third from the final set {C} is 1.0.
Calculate the total log-probability (using the natural logarithm, ln) for the complete ranked sequence A > B > C. Provide your final answer rounded to two decimal places and show the summation formula you used.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Set of Remaining Items in a Ranked Sequence
Plackett-Luce Loss Function
A model is designed to rank a set of three documents {Doc A, Doc B, Doc C} for a given user query. To calculate the log-probability of the specific ranked sequence 'Doc A > Doc B > Doc C', a developer proposes calculating the total log-probability as the sum of the log-probabilities of each document being chosen first from the full set of three documents. Why is this approach fundamentally flawed for modeling a sequential ranking process?
Ranked Sequence Log-Probability Calculation
Calculating Log-Probability for a Ranked List