Self-consistency can be interpreted as a search process guided by the principle of minimum Bayes risk. The objective of this process is to select the best possible output from a pool of candidates by finding the one that minimizes the overall Bayes risk.

Self-Consistency as a Minimum Bayes Risk Search Process

A language model is tasked with answering the question, 'What is the capital of Australia?'. It generates the four candidate answers listed in the case study below. Explain how the process of selecting the single best answer from this set can be framed as a search problem. In your explanation, identify what constitutes the 'search space' and propose a plausible 'evaluation metric' that could be used to find the optimal solution.

Framing Answer Selection as a Search Problem

An LLM generates five different step-by-step solutions to a complex algebra problem. A separate verification model then evaluates each solution by checking if the final answer is correct and if each intermediate step logically follows from the previous one. The solution with the highest score from the verifier is chosen as the final output. Match the components of this process, when framed as a search problem, to their correct descriptions.

An AI-powered coding assistant is tasked with writing a function to sort a list of numbers. It generates three different implementations of the sorting algorithm. To select the best one, it runs a series of tests on each implementation, measuring its execution speed and memory usage. The implementation that is fastest while using the least memory is chosen. Analyze this entire process by framing it as a search problem. In your analysis, identify and describe the following components: 1) the search space, 2) the evaluation metric used to define the 'best' solution, and 3) the search strategy.

Analyzing Code Generation as a Search Problem

Best-of-N sampling is a method used to improve model outputs by generating multiple (N) potential solutions for a given input. From this set of N outputs, the best one is selected using a specific mechanism, such as a verifier model or an evaluation function. This technique leverages a model's ability to generate diverse responses to find a higher-quality solution than what might be produced in a single attempt.

Best-of-N Sampling

Parallel scaling, also known as best-of-N sampling, is a strategy that involves generating K independent candidate solutions by running a base LLM multiple times. During this generation process, the sampling temperature can be adjusted to control the diversity of the outputs. After the candidates are created, a verifier evaluates each of the K complete solutions, and the one with the highest score is selected as the final answer. This method is conceptually analogous to using a reward model to select the best option from a set of sampled outputs.

Best-of-N Sampling (Parallel Scaling)

Within the framework of treating solution selection as a search problem, a search algorithm defines the strategy used to explore the space of possible output sequences and generate a set of candidate solutions. These strategies can range from simple independent sampling to more sophisticated algorithmic search techniques.

Search Algorithm for Solution Selection

The process of evaluating multiple potential output sequences and selecting the best one can be framed as a search problem. This framework involves two main components: a search algorithm that explores the space of possible output sequences to generate a set of candidate solutions, and a verifier that evaluates these candidates to pick the optimal result.

Google

The fundamental principle of the predict-then-verify approach is that for a given input, such as a math problem, a model can generate multiple potential output sequences or solutions. A separate verifier or selection mechanism then evaluates each of these generated solutions to identify and select the best one. This entire selection process can be framed as a search problem. Best-of-N sampling is a key example of this method.

Predict-then-Verify Approaches in LLM Reasoning

Reference of Foundations of Large Language Models Course

A core challenge in predict-then-verify approaches is the evaluation of an LLM's reasoning results or intermediate steps. To address this, 'verifiers' have been developed and implemented. These are specialized components designed to assess and validate the outputs generated during the LLM's reasoning process.

Verifiers in LLM Reasoning

The 'predict-then-refine' paradigm is a long-standing concept in Natural Language Processing where an initial output is generated and then iteratively improved. This approach has historical roots, with early examples including rule-based systems like Brill's tagger, and continues to be relevant in the modern deep learning era in various sequence-to-sequence tasks.

The Predict-then-Refine Paradigm in NLP

Self-refinement is an iterative framework where a solution is progressively improved through repeated cycles of generation, evaluation (critique), and revision. This process continues until the solution's quality, as assessed by a verifier, converges to a satisfactory level.

Self-Refinement in LLMs

This is a method for tackling complex reasoning tasks by directing a Large Language Model to first produce potential reasoning steps or 'thinking paths' and then to evaluate the validity of these paths.

Generating and Verifying Thinking Paths

Solution Selection as a Search Problem

A reasoning path is a structured, step-by-step method for solving a problem. For a given input problem, `x`, the solution is formed by a sequence of intermediate reasoning steps, `a_1, a_2, ...`, creating a logical chain. This progression can be visualized as a sequence, such as: `Problem (x) Step 1 (a1) Step 2 (a2) Step 3 (a3) ... Solution`.

Reasoning Path in Problem Solving

Parallel scaling and self-refinement (sequential scaling) differ fundamentally in their approach to generating solutions. Parallel scaling involves generating multiple independent solutions concurrently from the same initial problem, creating a set of distinct options from which a verifier selects the best one. In contrast, self-refinement is a sequential and iterative process that creates a single lineage of solutions. It starts with an initial solution, which is then progressively improved through cycles of feedback and revision, transforming one solution into the next. The verifier in self-refinement actively guides this evolution, rather than simply selecting a final answer from a static set.

Comparison of Parallel Scaling and Self-Refinement

A verifier is a model or function responsible for evaluating the quality, correctness, or utility of each candidate solution generated by a search algorithm. It provides a judgment, such as a score or probability, that allows the system to select the best candidate. The verifier can be implemented as another Large Language Model (LLM) or as a set of predefined rules or heuristics.

Verifier

For a given input in a reasoning task, the generated solution $${}\mathbf{y}$$ can be formalized as a sequence of intermediate reasoning steps, expressed mathematically as $${}\mathbf{y} = \{\bar{\mathbf{y}}_1,...,\bar{\mathbf{y}}_{n_s}\}$. The actual conclusion of the reasoning process is assumed to be located in the final step, $${}\bar{\mathbf{y}}_{n_s}$$, which can then be easily verified for correctness.

Solution as a Sequence of Reasoning Steps

A team is developing a system to solve complex mathematical word problems using a large language model. Their goal is to maximize the final answer's accuracy. Which of the following strategies best exemplifies a process where multiple potential solutions are first generated and then evaluated to select the most reliable one?

Analyze the two systems described in the case study below. Explain the fundamental principle that accounts for System B's higher reliability compared to System A, even when both systems use a similar underlying generative model.

Analyzing LLM Reasoning Strategies

A system is designed to solve a complex problem by first generating multiple possible answers and then selecting the best one. Arrange the following steps to accurately represent this two-stage workflow.

In a system designed to solve a problem by first generating multiple potential solutions and then using a separate component to select the best one, the quality of the final selected answer depends solely on the generative capability of the initial model.

You are reviewing a proposed architecture for an i...

You’re designing an internal LLM assistant for a f...

You’re leading an internal rollout of an LLM assis...

In an LLM-based customer support assistant, the mo...

You are reviewing a proposed architecture for an internal LLM assistant used by Finance Operations to (1) draft a vendor-payment approval note and (2) optionally trigger an external API call `create_payment(vendor_id, amount, invoice_id)` that will schedule a real payment. The team has observed two failure modes: (a) the model sometimes hallucinates invoice details when the user’s message is incomplete, and (b) when the model does call the API, it occasionally chooses the wrong invoice_id among several similar open invoices.

Write a design critique and improvement plan that integrates: (i) deliberate-then-generate prompting (the model must first surface likely error types/uncertainties before drafting the approval note), (ii) a predict-then-verify strategy that generates multiple candidate action plans (including whether to call the API at all) and selects among them, (iii) an explicit verifier component (describe what it checks and whether it is outcome-based, process-based, or both), and (iv) safe tool-use with the external API (describe gating, required arguments, and what happens when required data is missing).

In your answer, explain the tradeoffs you are making (latency, cost, and risk), and give at least two concrete examples of verifier checks that would specifically reduce the two observed failure modes without relying on the model’s pre-trained knowledge alone.

Design Review: Combining Tool Use, DTG, and Predict-then-Verify for a High-Stakes API Workflow

You are designing an internal LLM assistant for a finance operations team. A user asks: “Can I approve this vendor payment today? If not, what exactly is blocking it and what should I do next?” The correct answer depends on real-time data from two internal systems exposed via APIs: (1) an invoice/PO matching service and (2) a sanctions/AML screening service. The business requires (a) high accuracy, (b) an auditable rationale, and (c) minimal latency/cost.

Write an essay proposing a single end-to-end inference workflow that combines: (i) tool use with external APIs, (ii) a deliberate-then-generate step that surfaces likely error modes before drafting the final response, (iii) a predict-then-verify strategy that generates multiple candidate decisions/explanations, (iv) a verifier that selects or rejects candidates, and (v) a self-reflection step that decides whether to call additional tools or revise the answer.

In your proposal, be explicit about: what the model generates at each stage, when API calls happen, what the verifier checks (and what it cannot guarantee), how self-reflection changes control flow, and the key tradeoffs you are making among accuracy, auditability, and latency/cost. Assume the APIs can occasionally return incomplete data or transient errors.

Designing a Reliable LLM Workflow for Real-Time Decisions

You lead an LLM platform team. A customer-facing assistant can call two internal APIs: (1) `get_account_balance(account_id)` and (2) `get_recent_transactions(account_id, days)`. Last week, the assistant told a customer they had “$0 available” and recommended a payment plan. An audit later showed the model (a) called the correct APIs but (b) misread a negative pending authorization as the final balance and (c) produced a confident explanation that sounded plausible. You are asked to propose a revised inference-time workflow that reduces the chance of this kind of error without adding more than ~1 second median latency.

Write an essay that (i) designs a concrete end-to-end flow combining deliberate-then-generate, self-reflection, and a predict-then-verify stage; (ii) specifies what the verifier checks and whether it should be outcome-based, process-based, or a hybrid; and (iii) explains how and when the model should use the external APIs (including what to do when API outputs are ambiguous or inconsistent). Your answer must explicitly discuss the tradeoffs among accuracy, latency, and failure modes (e.g., false rejects vs false accepts), and justify why your design would have prevented the incident described.

Post-Incident Analysis: Preventing Confidently Wrong API-Backed Answers

You are the product owner for an internal LLM assistant used by Customer Operations to answer: (1) “Where is order #12345 right now?” and (2) “Can I promise delivery by Friday?” The assistant can call two external APIs: `get_tracking(order_id)` (returns latest scan location + timestamp) and `get_inventory(sku, warehouse)` (returns available-to-promise quantity). A recent incident occurred: the assistant confidently promised Friday delivery based on a stale tracking scan and an incorrect assumption about inventory allocation rules. Leadership now requires: (a) fewer than 2 API calls per user request on average, (b) a measurable reduction in incorrect commitments, and (c) an auditable record of why the assistant made a commitment.

As the designer, propose a single end-to-end inference workflow that integrates: deliberate analysis before answering, self-reflection, predict-then-verify with a verifier, and tool use with the APIs above. Your answer must specify (i) when and why the model calls each API (or chooses not to), (ii) what the model generates as multiple “candidates” (what varies across candidates), (iii) what the verifier checks and what evidence it uses (including how it handles stale timestamps), and (iv) what the final user-facing response should contain to be auditable while minimizing overconfident promises.

Case Study: Shipping a Tool-Using LLM Assistant with Built-In Verification Under Latency Constraints

You are reviewing an internal pilot of an LLM-powered customer support assistant for a subscription product. The assistant can call two external APIs:

- `get_invoice(customer_id, invoice_id)` → returns line items, taxes, discounts, currency, and current payment status.
- `create_refund(invoice_id, amount, currency, reason)` → executes a refund immediately and returns a refund confirmation ID.

Incident: A customer asked, “I was double-charged on invoice INV-8841—refund the extra charge.” The assistant responded confidently: “You were charged twice; I’ve refunded $49.99,” and then called `create_refund(INV-8841, 49.99, "USD", "duplicate charge")`. Later, finance found the invoice was in EUR, the ‘double charge’ was actually an authorization + capture, and the correct action was to provide an explanation (no refund). The team wants a redesign that (1) minimizes extra model calls/latency, (2) reduces the chance of executing an incorrect refund, and (3) still uses the LLM to handle ambiguous customer language.

As the reviewer, propose a single end-to-end workflow (not a list of unrelated tips) that integrates: (a) a deliberate-then-generate step, (b) a predict-then-verify mechanism with an explicit verifier, (c) self-reflection to catch overconfident claims, and (d) safe external API tool use. Your answer must specify where in the flow the model generates candidates, what the verifier checks (outcome vs. step-level), what evidence must be pulled from `get_invoice`, and the exact gating rule that prevents `create_refund` from being called when uncertainty or mismatches (e.g., currency/status) are detected.

Case Review: Preventing Incorrect Refund Commitments in an LLM + Payments API Assistant

You are designing an internal LLM assistant used by procurement and security teams to draft vendor risk review summaries. The assistant can call two internal APIs during inference: (1) `get_vendor_certifications(vendor_id)` which returns a list of current certifications with expiry dates, and (2) `get_latest_security_incidents(vendor_id)` which returns incident summaries from the last 12 months. A recent near-miss occurred: the assistant confidently wrote, “Vendor X is SOC 2 Type II certified through next year and has had no security incidents,” but the certification had expired two months earlier and there was a medium-severity incident last quarter. Leadership requires a redesign that (a) reduces the chance of false claims, (b) keeps median response time under 8 seconds, and (c) produces an auditable trail showing why the final statements were made.

Propose a single end-to-end inference workflow (not training) that integrates: deliberate-then-generate self-reflection, predict-then-verify with a verifier, and tool use with the APIs above. Your answer must specify (i) when and how many candidate drafts are generated, (ii) what the verifier checks (and whether it is outcome-based, process-based, or both), (iii) how API results are used to ground or block claims, and (iv) how the workflow meets the 8-second latency constraint while still improving reliability. Be concrete about the sequence of steps and the key tradeoffs you are making.

Case Study: Preventing Hallucinated Compliance Claims in an API-Enabled LLM for Vendor Risk Reviews

Sequential scaling, also referred to as self-refinement, builds a sequence of solutions incrementally. It starts with an initial solution generated by a Large Language Model. Then, a verifier (often the same model) evaluates the solution in a critique stage to produce feedback, such as textual critiques, numerical scores, or revised plans. In the refine stage, the model uses the original problem, the current solution, and this feedback to generate a potentially improved solution. This critique-refine cycle is repeated iteratively, allowing the verifier to actively guide the generation process instead of simply selecting the best outcome from a static set of candidates.

Learn Before

Related

Learn After