Formula for the Risk of an Output in Minimum Bayes Risk Decoding
The total risk of a candidate output within a set of potential outputs is calculated as the expected value of the risk function over all other outputs . This expectation is weighted by the conditional probability of each output given the input . The formula is:
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Risk Function for Minimum Bayes Risk Decoding
Formula for the Risk of an Output in Minimum Bayes Risk Decoding
A system generates five potential answers to a question. The goal is to select the single best answer by choosing the one that minimizes the total expected cost when compared against all other generated answers. The cost of selecting answer 'y' over an alternative 'y_r' is defined as 0 if they are identical and 1 if they are different. Given the following set of generated answers, which one should be selected?
Generated Answers:
- 'Paris'
- 'Lyon'
- 'Paris'
- 'Paris'
- 'Marseille'
A developer implements a system where a language model generates 10 possible answers to a single prompt. The system then selects the most frequently occurring answer as the final output. Which of the following statements best analyzes this selection strategy from a theoretical perspective?
Evaluating Answer Selection Strategies
Learn After
Calculating Output Risk in a Generation Task
A system calculates the total risk for a candidate output 'y' against a set of reference outputs 'yr' using the formula: Risk(y) = Σ [R(y, yr) * Pr(yr|x)], where R(y, yr) is a risk function and Pr(yr|x) is the probability of a reference output. Assume the risk function R(y, yr) returns a value of 1 if y is different from yr, and 0 otherwise. If the probability, Pr(yr|x), of a specific reference output 'yr' that is different from the candidate 'y' increases, what is the direct impact on the calculated Risk(y)?
Role of Probability Weighting in Risk Calculation