Comparison of Execution Timing in Tool Use and RAG

Limitation of Pre-trained LLMs in Tool Use

Using Web Search as an External Tool for LLMs

Application of LLMs to Mathematical Problems

A development team is building a chatbot for an airline. The chatbot must be able to answer user questions like, 'What is the status of flight UA456?' and 'Are there any business class seats available on the 10 AM flight to London tomorrow?'. The airline's flight data is stored in a private, real-time database. Which of the following represents the most effective and reliable approach for the team to implement these features?

A user asks a Large Language Model, 'What is the capital of Brazil and what is the current time there?'. The model has access to an external tool get_current_time(city). Arrange the following steps in the logical order the model would follow to answer the user's request.

Troubleshooting an LLM-Powered Sales Assistant

Designing a Reliable LLM Workflow for Real-Time Decisions

Design Review: Combining Tool Use, DTG, and Predict-then-Verify for a High-Stakes API Workflow

Post-Incident Analysis: Preventing Confidently Wrong API-Backed Answers

Case Review: Preventing Incorrect Refund Commitments in an LLM + Payments API Assistant

Case Study: Shipping a Tool-Using LLM Assistant with Built-In Verification Under Latency Constraints

Case Study: Preventing Hallucinated Compliance Claims in an API-Enabled LLM for Vendor Risk Reviews

You are reviewing a proposed architecture for an i...

In an LLM-based customer support assistant, the mo...

You’re designing an internal LLM assistant for a f...

You’re leading an internal rollout of an LLM assis...

Methods for Activating Self-Reflection in LLMs

An AI model is asked, 'What is the approximate distance from the Earth to the Moon?' It provides two consecutive responses:

• Response 1: 'The distance from the Earth to the Moon is about 238,900 kilometers.'
• Response 2: 'Upon review, my previous answer was imprecise. The distance is in miles, not kilometers. The correct average distance is approximately 238,900 miles, which is about 384,400 kilometers. Stating the unit correctly is crucial for accuracy.'

Which of the following best analyzes the process demonstrated in Response 2?

Evaluating AI Response Quality

Mechanism of AI Self-Correction

You are reviewing a proposed architecture for an i...

You’re designing an internal LLM assistant for a f...

You’re leading an internal rollout of an LLM assis...

In an LLM-based customer support assistant, the mo...

Design Review: Combining Tool Use, DTG, and Predict-then-Verify for a High-Stakes API Workflow

Designing a Reliable LLM Workflow for Real-Time Decisions

Post-Incident Analysis: Preventing Confidently Wrong API-Backed Answers

Case Study: Shipping a Tool-Using LLM Assistant with Built-In Verification Under Latency Constraints

Case Review: Preventing Incorrect Refund Commitments in an LLM + Payments API Assistant

Case Study: Preventing Hallucinated Compliance Claims in an API-Enabled LLM for Vendor Risk Reviews

Limitation of the Deliberate-then-Generate (DTG) Method

Comparison of Iterative vs. Non-Iterative Prompting Methods

Instructional Component of the DTG Prompt Template for Translation Refinement

Integration of Feedback and Refinement in the DTG Method

A developer is using a Large Language Model to refine a technical summary. They want the model to first identify any factual inaccuracies or unclear statements in the original text and then, based on that analysis, produce a corrected and more coherent version. Which of the following approaches correctly implements the 'Deliberate-then-Generate' method for this task?

Input Structure of the DTG Prompt for Chinese-to-English Translation

Challenge of LLM-Based Error Identification in Translation

A developer is designing a workflow to refine user-generated reports using a Large Language Model. The primary goal is to ensure the model first analyzes potential issues (e.g., ambiguity, factual errors) before rewriting the report, all while minimizing the number of interactions with the model. Which of the following prompt structures best represents the 'Deliberate-then-Generate' method for this task?

Analysis of a Translation Refinement Process

You are reviewing a proposed architecture for an i...

You’re designing an internal LLM assistant for a f...

You’re leading an internal rollout of an LLM assis...

In an LLM-based customer support assistant, the mo...

Design Review: Combining Tool Use, DTG, and Predict-then-Verify for a High-Stakes API Workflow

Designing a Reliable LLM Workflow for Real-Time Decisions

Post-Incident Analysis: Preventing Confidently Wrong API-Backed Answers

Case Study: Shipping a Tool-Using LLM Assistant with Built-In Verification Under Latency Constraints

Case Review: Preventing Incorrect Refund Commitments in an LLM + Payments API Assistant

Case Study: Preventing Hallucinated Compliance Claims in an API-Enabled LLM for Vendor Risk Reviews

Supervised Learning of Verifiers

Relation between Verifiers and RLHF Reward Models

Classification of Verification Approaches

Guiding Role of the Verifier in Self-Refinement

A system is designed to solve complex, multi-step logic puzzles. First, a generative model produces five different potential step-by-step solutions to a given puzzle. Then, a second, distinct component is used. This second component's sole function is to evaluate each of the five proposed solutions by scoring the logical soundness of each step in the reasoning chain. Based on these scores, it selects the single most coherent and valid solution to present as the final answer. What is the primary role of this second component in the system's architecture?

Improving an AI Tutoring System

Consider a system that solves a problem by first having one component generate several different step-by-step solutions. For this system to be effective, the same component that generated the solutions must also be used to evaluate them and select the best one.

You are reviewing a proposed architecture for an i...

You’re designing an internal LLM assistant for a f...

You’re leading an internal rollout of an LLM assis...

In an LLM-based customer support assistant, the mo...

Design Review: Combining Tool Use, DTG, and Predict-then-Verify for a High-Stakes API Workflow

Designing a Reliable LLM Workflow for Real-Time Decisions

Post-Incident Analysis: Preventing Confidently Wrong API-Backed Answers

Case Study: Shipping a Tool-Using LLM Assistant with Built-In Verification Under Latency Constraints

Case Review: Preventing Incorrect Refund Commitments in an LLM + Payments API Assistant

Case Study: Preventing Hallucinated Compliance Claims in an API-Enabled LLM for Vendor Risk Reviews

Verifiers in LLM Reasoning

The Predict-then-Refine Paradigm in NLP

Self-Refinement in LLMs

Generating and Verifying Thinking Paths

Solution Selection as a Search Problem

Reasoning Path in Problem Solving

Best-of-N Sampling (Parallel Scaling)

Comparison of Parallel Scaling and Self-Refinement

Verifier

Solution as a Sequence of Reasoning Steps

A team is developing a system to solve complex mathematical word problems using a large language model. Their goal is to maximize the final answer's accuracy. Which of the following strategies best exemplifies a process where multiple potential solutions are first generated and then evaluated to select the most reliable one?

Analyzing LLM Reasoning Strategies

A system is designed to solve a complex problem by first generating multiple possible answers and then selecting the best one. Arrange the following steps to accurately represent this two-stage workflow.

In a system designed to solve a problem by first generating multiple potential solutions and then using a separate component to select the best one, the quality of the final selected answer depends solely on the generative capability of the initial model.

You are reviewing a proposed architecture for an i...

You’re designing an internal LLM assistant for a f...

You’re leading an internal rollout of an LLM assis...

In an LLM-based customer support assistant, the mo...

Design Review: Combining Tool Use, DTG, and Predict-then-Verify for a High-Stakes API Workflow

Designing a Reliable LLM Workflow for Real-Time Decisions

Post-Incident Analysis: Preventing Confidently Wrong API-Backed Answers

Case Study: Shipping a Tool-Using LLM Assistant with Built-In Verification Under Latency Constraints

Case Review: Preventing Incorrect Refund Commitments in an LLM + Payments API Assistant

Case Study: Preventing Hallucinated Compliance Claims in an API-Enabled LLM for Vendor Risk Reviews

Sequential Scaling