Learn Before
Automated Code Generation and Selection
Based on the scenario, propose a component that could be added to the system to automatically select the most efficient code snippet. Describe the specific function of this component and the type of output it would need to generate to make a final selection.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Using a Verifier to Score and Select Candidates
Off-the-Shelf Tools as Verifiers
Using a Large Language Model as a Verifier
Heuristic-Based Verifiers
Final-Answer Verification
Automated Code Generation and Selection
A system is designed to solve complex math word problems. First, a language model generates five different step-by-step solutions for a given problem. Next, a separate component examines each of the five solutions, checks the final numerical answer for correctness against a known calculator result, and assigns a 'correctness score' to each. The solution with the highest score is then presented as the final answer. Which part of this system is acting as the verifier?
Best-of-N Sampling (Parallel Scaling)
Evaluating a Verifier for Factual Summarization