Risk Function for Minimum Bayes Risk Decoding
In the Minimum Bayes Risk (MBR) framework, the risk function, denoted as , is a core component defined for every pair of outputs. It quantifies the penalty or cost incurred when one output, , is chosen as the final answer instead of another potential output, .
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Risk Function for Minimum Bayes Risk Decoding
Formula for the Risk of an Output in Minimum Bayes Risk Decoding
A system generates five potential answers to a question. The goal is to select the single best answer by choosing the one that minimizes the total expected cost when compared against all other generated answers. The cost of selecting answer 'y' over an alternative 'y_r' is defined as 0 if they are identical and 1 if they are different. Given the following set of generated answers, which one should be selected?
Generated Answers:
- 'Paris'
- 'Lyon'
- 'Paris'
- 'Paris'
- 'Marseille'
A developer implements a system where a language model generates 10 possible answers to a single prompt. The system then selects the most frequently occurring answer as the final output. Which of the following statements best analyzes this selection strategy from a theoretical perspective?
Evaluating Answer Selection Strategies
Learn After
Calculating Risk for a Candidate Sentence
A team is developing a text summarization system. The primary goal is to generate summaries that are factually consistent with the source document, even if it means the summary is less fluent or grammatically perfect. The system generates multiple candidate summaries and must select the best one by minimizing the expected penalty against all other candidates. Which of the following penalty functions, R(y, y_r), which measures the cost of choosing candidate summary 'y' over reference summary 'y_r', would be most appropriate for this specific goal?
A crucial aspect of selecting the best output from a set of candidates is defining a function, R(y, y_r), that measures the penalty of selecting a candidate output 'y' over another potential output 'y_r'. Match each description of a penalty function to the primary type of error it is designed to penalize.