Risk Function in Minimum Bayes Risk Decoding
In the context of minimum Bayes risk, a risk function, denoted as , is defined for every pair of candidate outputs. This function quantifies the penalty or cost associated with replacing a chosen output with an alternative reference output .
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Risk Function in Minimum Bayes Risk Decoding
Risk of an Output in Minimum Bayes Risk Decoding
A language model is prompted to solve the math problem 'What is 7 + 8?'. To improve reliability, the model generates five different outputs using a sampling strategy: [15, 14, 15, 15, 16]. A selection process is then used to choose the final answer by identifying the candidate that minimizes the expected disagreement with the other generated candidates. Which output will be selected?
Connecting Self-Consistency to a Formal Framework
Evaluating a Text Generation Strategy
Learn After
Risk of an Output in Minimum Bayes Risk Decoding
A language model is generating a summary for a document. The ideal, human-written reference summary is
yr: "The study found a significant link between diet and health."The model considers two candidate summaries:
y1: "The study found a significant link between diet and health."y2: "Research showed a connection between food and wellness."
Given the purpose of a risk function,
R(candidate, reference), which of the following statements best describes how it would operate in this scenario?Evaluating a Simple Risk Function
Evaluating a Risk Function for Legal Translation