Learn Before
Self-Consistency as a Minimum Bayes Risk Search Process
Self-consistency can be interpreted as a search process guided by the principle of minimum Bayes risk. The objective of this process is to select the best possible output from a pool of candidates by finding the one that minimizes the overall Bayes risk.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Related
Self-Consistency as a Minimum Bayes Risk Search Process
Framing Answer Selection as a Search Problem
An LLM generates five different step-by-step solutions to a complex algebra problem. A separate verification model then evaluates each solution by checking if the final answer is correct and if each intermediate step logically follows from the previous one. The solution with the highest score from the verifier is chosen as the final output. Match the components of this process, when framed as a search problem, to their correct descriptions.
Analyzing Code Generation as a Search Problem
Best-of-N Sampling
Best-of-N Sampling (Parallel Scaling)
Search Algorithm for Solution Selection
Learn After
Risk Function for Minimum Bayes Risk Decoding
Formula for the Risk of an Output in Minimum Bayes Risk Decoding
A system generates five potential answers to a question. The goal is to select the single best answer by choosing the one that minimizes the total expected cost when compared against all other generated answers. The cost of selecting answer 'y' over an alternative 'y_r' is defined as 0 if they are identical and 1 if they are different. Given the following set of generated answers, which one should be selected?
Generated Answers:
- 'Paris'
- 'Lyon'
- 'Paris'
- 'Paris'
- 'Marseille'
A developer implements a system where a language model generates 10 possible answers to a single prompt. The system then selects the most frequently occurring answer as the final output. Which of the following statements best analyzes this selection strategy from a theoretical perspective?
Evaluating Answer Selection Strategies