Learn Before
Framing Answer Selection as a Search Problem
A language model is tasked with answering the question, 'What is the capital of Australia?'. It generates the four candidate answers listed in the case study below. Explain how the process of selecting the single best answer from this set can be framed as a search problem. In your explanation, identify what constitutes the 'search space' and propose a plausible 'evaluation metric' that could be used to find the optimal solution.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Self-Consistency as a Minimum Bayes Risk Search Process
Framing Answer Selection as a Search Problem
An LLM generates five different step-by-step solutions to a complex algebra problem. A separate verification model then evaluates each solution by checking if the final answer is correct and if each intermediate step logically follows from the previous one. The solution with the highest score from the verifier is chosen as the final output. Match the components of this process, when framed as a search problem, to their correct descriptions.
Analyzing Code Generation as a Search Problem
Best-of-N Sampling
Best-of-N Sampling (Parallel Scaling)
Search Algorithm for Solution Selection