Direct Conclusion Generation with Hidden Reasoning

Single-Run Multi-Step Reasoning

Multi-Run Problem Decomposition for Complex Reasoning

Self-Refinement in LLMs

Predict-then-Verify Approaches in LLM Reasoning

Principle of Generating Longer Reasoning Paths

Modifying Decoding for Longer Reasoning Paths

Multi-Stage Generation for Incremental Reasoning

An engineer is building a system to solve complex logic puzzles. When a puzzle is submitted, the system sends a single, carefully crafted prompt to a large language model. The model's output is a complete, step-by-step explanation of how it solved the puzzle, followed by the final answer, all generated in one response. Which approach to multi-step reasoning does this system exemplify?

Prompting for a Reasoning Process to Mitigate Errors in Complex Tasks

Compositional Generalization in LLMs

Choosing a Reasoning Strategy for a Financial AI

You are designing systems that use a large language model to solve complex problems. Match each system description with the reasoning approach it employs.

Challenging Reasoning Tasks for LLMs

Self-Refinement in LLMs

Model Ensembling for Text Generation

Output Ensembling

Retrieval-Augmented Generation (RAG)

LLM Tool Use with External APIs

Evolution of the Concept of Alignment in NLP

Analyze the two scenarios below, each showing an incorrect output from a language model. Which scenario provides the clearest example of a failure caused by the model's lack of implicit knowledge, rather than a simple factual error in its training data?

Analyzing an LLM's Reasoning Failure

Limitations of Pre-trained Knowledge in Standard LLMs

Explaining an LLM's Reasoning Error

Verifiers in LLM Reasoning

The Predict-then-Refine Paradigm in NLP

Self-Refinement in LLMs

Generating and Verifying Thinking Paths

Solution Selection as a Search Problem

Reasoning Path in Problem Solving

Best-of-N Sampling (Parallel Scaling)

Comparison of Parallel Scaling and Self-Refinement

Verifier

Solution as a Sequence of Reasoning Steps

A team is developing a system to solve complex mathematical word problems using a large language model. Their goal is to maximize the final answer's accuracy. Which of the following strategies best exemplifies a process where multiple potential solutions are first generated and then evaluated to select the most reliable one?

Analyzing LLM Reasoning Strategies

A system is designed to solve a complex problem by first generating multiple possible answers and then selecting the best one. Arrange the following steps to accurately represent this two-stage workflow.

In a system designed to solve a problem by first generating multiple potential solutions and then using a separate component to select the best one, the quality of the final selected answer depends solely on the generative capability of the initial model.

You are reviewing a proposed architecture for an i...

You’re designing an internal LLM assistant for a f...

You’re leading an internal rollout of an LLM assis...

In an LLM-based customer support assistant, the mo...

Design Review: Combining Tool Use, DTG, and Predict-then-Verify for a High-Stakes API Workflow

Designing a Reliable LLM Workflow for Real-Time Decisions

Post-Incident Analysis: Preventing Confidently Wrong API-Backed Answers

Case Study: Shipping a Tool-Using LLM Assistant with Built-In Verification Under Latency Constraints

Case Review: Preventing Incorrect Refund Commitments in an LLM + Payments API Assistant

Case Study: Preventing Hallucinated Compliance Claims in an API-Enabled LLM for Vendor Risk Reviews

Sequential Scaling

Brill's Tagger as an Early Example of Predict-then-Refine

Modern NLP Applications of the Predict-then-Refine Paradigm

Self-Refinement in LLMs

An AI-powered code completion tool is designed to help developers write functions. When a developer provides a function name and a comment describing its purpose, the tool first generates a complete, functional block of code. Following this initial generation, the tool enters a loop where it analyzes the code it just wrote, identifies potential inefficiencies or non-standard practices, and applies a specific correction. This analysis-and-correction loop repeats several times, with the code block being progressively improved at each step. Which statement accurately characterizes the fundamental approach this tool uses?

Distinguishing NLP System Architectures

Analyzing System Architectures for Output Generation

Sequential Scaling