Preference Probability Calculation
An AI development team is refining a language model using a reward function, r(prompt, response), which assigns a quality score to a given response. The probability that a preferred response (y_a) is chosen over another response (y_b) is determined by the formula: Pr(y_a ≻ y_b | prompt) = Sigmoid(r(prompt, y_a) - r(prompt, y_b)), where Sigmoid(z) = 1 / (1 + e^-z). Given the scenario below, calculate the probability that Response A is preferred over Response B. Show your calculation and provide the final probability rounded to two decimal places.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Listwise Loss Formula from Accumulated Pairwise Comparisons
Empirical Reward Model Loss Formula
Empirical Formulation of Pair-wise Ranking Loss
A system learns a function,
r(input, response), that assigns a numerical score indicating the quality of aresponsefor a giveninput. The probability that responseY_ais preferred over responseY_bis then calculated using the formula:Probability = Sigmoid(r(input, Y_a) - r(input, Y_b)), whereSigmoid(z) = 1 / (1 + e^-z). Given the following scenarios for a single input, which one presents a logical inconsistency between the assigned scores and the resulting preference probability?Preference Probability Calculation
Invariance of Preference Probability