Stabilizing a Decomposition-Based LLM Workflow for a Regulated Customer-Email Triage System
You are leading an internal team deploying an LLM to triage inbound customer emails for a regulated financial-services product. The system must (1) classify each email into one of five queues (Billing, Technical Issue, Account Access, Complaint, Potential Fraud), (2) extract any required follow-up questions to ask the customer, and (3) produce a short agent-facing rationale that cites only information present in the email. The current design uses a fixed list of sub-problems generated up front and then solved in order. In production, you observe two recurring failures: (a) the model asks irrelevant follow-up questions because it forgets constraints discovered earlier (e.g., it already determined the customer is locked out but later asks for billing details), and (b) some emails contain nested issues (e.g., a complaint that includes a potential fraud indicator) that the fixed decomposition misses, leading to misrouting.
As the owner, propose a revised end-to-end LLM interaction plan that addresses both failures by combining: (i) how you will generate sub-problems, (ii) how you will solve them sequentially while preserving intermediate results, and (iii) when and how you will trigger recursive decomposition for a sub-problem that turns out to be too complex or multi-faceted. Your answer must include at least one concrete example of a contextual QA pair you would carry forward and explain how it changes the next sub-problem’s prompt or decision.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Divide-and-Conquer Paradigm
Example of a Classification Task for LLMs: Identifying AI Risks in a Document
Approaches to Multi-Step Reasoning in LLMs
Two-Step Problem Decomposition
Dynamic Problem Decomposition for Complex Reasoning
Compositionality in NLP
Outlining as a Method of Problem Decomposition for Generative Tasks
General Framework of Problem Decomposition
A team is using a large language model to automate complex tasks. They decide to implement a strategy where a main problem is broken down into a complete, fixed list of sub-problems before the model begins to solve any of them. The model will then solve each sub-problem in sequence. For which of the following tasks is this pre-defined decomposition approach LEAST likely to succeed?
Evaluating a Problem Decomposition Strategy for Multi-Hop QA
Illustrating the Need for Decomposition in Generative Tasks
Complex Reasoning Problems
Multi-hop Question Answering
A development team is building several applications powered by a large language model. Match each application's primary task with the most suitable strategy for breaking down the problem.
Designing a Decomposition-Driven LLM Workflow for a High-Stakes Corporate Task
Debugging a Decomposition-Based LLM Workflow Using Recursive Sub-Problems and Contextual QA Pairs
Evaluating and Redesigning a Decomposition Workflow Under Context and Cost Constraints
Designing a Decomposition-and-QA-Pair Workflow for Contract Review with Recursive Escalation
Stabilizing a Decomposition-Based LLM Workflow for a Regulated Customer-Email Triage System
Designing a Decomposition Workflow for Root-Cause Analysis of a Production Incident
Create a Recursive, Context-Carrying Decomposition Plan for LLM-Assisted KPI Narrative Generation
You are building an internal LLM assistant to answ...
You are designing an internal LLM workflow to answ...
You’re building an internal LLM workflow to answer...
Your team is rolling out an internal LLM assistant...
You’re building an internal LLM workflow to produc...
You’re building an internal LLM assistant to help ...
You’re leading an internal enablement team buildin...
Choosing and Justifying a Prompting Strategy Under Context and Quality Constraints
Designing a Prompting Workflow for a High-Stakes, Multi-Step Task
Diagnosing and Redesigning a Prompting Approach for a Decomposed Workflow
Stabilizing an LLM Workflow for Multi-Step Policy Compliance Decisions
Debugging a Multi-Step LLM Workflow for Contract Clause Risk Triage
Designing a Robust Prompting Workflow for Multi-Step Root-Cause Analysis with Limited Examples
Psychological Perspective on Problem Decomposition
Tool Use as Problem Decomposition in LLMs
Decomposing a Complex Planning Task
A language model is tasked with generating a comprehensive report on the impact of remote work on employee productivity and well-being. Which of the following sets of sub-problems represents the most effective initial breakdown of this complex task to ensure a well-structured and complete final output?
Evaluating a Generated Problem Breakdown
You are building an internal LLM assistant to answ...
You are designing an internal LLM workflow to answ...
You’re building an internal LLM workflow to answer...
Create a Recursive, Context-Carrying Decomposition Plan for LLM-Assisted KPI Narrative Generation
Designing a Decomposition-Driven LLM Workflow for a High-Stakes Corporate Task
Evaluating and Redesigning a Decomposition Workflow Under Context and Cost Constraints
Debugging a Decomposition-Based LLM Workflow Using Recursive Sub-Problems and Contextual QA Pairs
Designing a Decomposition Workflow for Root-Cause Analysis of a Production Incident
Designing a Decomposition-and-QA-Pair Workflow for Contract Review with Recursive Escalation
Stabilizing a Decomposition-Based LLM Workflow for a Regulated Customer-Email Triage System
Dynamic Sub-Problem Generation
Sequential Sub-Problem Solving with Contextual QA Pairs
Expanding the Sub-Problem Solver Beyond LLMs
Recursive Decomposition of Sub-Problems
Framing Problem-Solving as a Reinforcement Learning Problem
An AI is tasked with creating a valid three-day weekend itinerary (Fri, Sat, Sun) to visit a museum, a park, and a specific restaurant. The AI first decomposes the problem and solves two sub-problems, yielding the following intermediate conclusions:
- The museum is only open on Friday and Saturday.
- The restaurant requires a reservation made at least one day in advance.
Which of the following statements best describes the next step in the sub-problem solving process to generate the final itinerary?
Synthesizing Sub-Problem Solutions
Analyzing a Flawed Project Plan
You are building an internal LLM assistant to answ...
You are designing an internal LLM workflow to answ...
You’re building an internal LLM workflow to answer...
Create a Recursive, Context-Carrying Decomposition Plan for LLM-Assisted KPI Narrative Generation
Designing a Decomposition-Driven LLM Workflow for a High-Stakes Corporate Task
Evaluating and Redesigning a Decomposition Workflow Under Context and Cost Constraints
Debugging a Decomposition-Based LLM Workflow Using Recursive Sub-Problems and Contextual QA Pairs
Designing a Decomposition Workflow for Root-Cause Analysis of a Production Incident
Designing a Decomposition-and-QA-Pair Workflow for Contract Review with Recursive Escalation
Stabilizing a Decomposition-Based LLM Workflow for a Regulated Customer-Email Triage System
Hierarchical Problem-Solving Structures
AI Task Management Strategy
An AI system is designed to plan a complex software project. It first breaks the project into three main tasks: 'Design Architecture', 'Develop Modules', and 'Test System'. The system successfully completes 'Design Architecture'. However, when it attempts 'Develop Modules', it gets stuck because this task itself consists of many interdependent sub-tasks like 'Develop UI', 'Develop Backend API', and 'Set up Database', each with its own complexities. What is the fundamental flaw in the system's problem-solving process?
An AI system is tasked with solving a complex problem. Arrange the following steps in the logical order that demonstrates a recursive approach to breaking down a sub-problem.
You are building an internal LLM assistant to answ...
You are designing an internal LLM workflow to answ...
You’re building an internal LLM workflow to answer...
Create a Recursive, Context-Carrying Decomposition Plan for LLM-Assisted KPI Narrative Generation
Designing a Decomposition-Driven LLM Workflow for a High-Stakes Corporate Task
Evaluating and Redesigning a Decomposition Workflow Under Context and Cost Constraints
Debugging a Decomposition-Based LLM Workflow Using Recursive Sub-Problems and Contextual QA Pairs
Designing a Decomposition Workflow for Root-Cause Analysis of a Production Incident
Designing a Decomposition-and-QA-Pair Workflow for Contract Review with Recursive Escalation
Stabilizing a Decomposition-Based LLM Workflow for a Regulated Customer-Email Triage System
Synthesizing Sub-Problem Answers for a Final Solution
Formula for Sequential Sub-Problem Solving
Prompting an LLM for the First Sub-Problem in a Sequence
Example of Constructing a Contextual Prompt for a Subsequent Sub-Problem
An AI assistant is tasked with planning a multi-day hiking trip. The task is broken down into a sequence of three steps. To solve the third and final step, 'Create a detailed meal plan,' two different approaches for providing the model with information are proposed.
- Approach X: The model is given the original task, the first question ('What is the weather forecast?') and its answer, the second question ('What gear is needed?') and its answer, and the final question ('Create a detailed meal plan.').
- Approach Y: The model is given only the second question ('What gear is needed?') and its answer, along with the final question ('Create a detailed meal plan.').
Which approach is more likely to produce a high-quality, contextually appropriate final answer, and why?
Constructing a Contextual Prompt for a Subsequent Sub-Problem
Synthesizing Sub-Problem Answers to Solve the Original Problem
An AI assistant is solving a complex problem by breaking it down into smaller, sequential questions. You need to construct the complete input that should be sent to the model to solve the third sub-problem. Arrange the following components in the correct logical order to create this input.
Diagnosing a Flawed AI Problem-Solving Process
You are building an internal LLM assistant to answ...
You are designing an internal LLM workflow to answ...
You’re building an internal LLM workflow to answer...
Create a Recursive, Context-Carrying Decomposition Plan for LLM-Assisted KPI Narrative Generation
Designing a Decomposition-Driven LLM Workflow for a High-Stakes Corporate Task
Evaluating and Redesigning a Decomposition Workflow Under Context and Cost Constraints
Debugging a Decomposition-Based LLM Workflow Using Recursive Sub-Problems and Contextual QA Pairs
Designing a Decomposition Workflow for Root-Cause Analysis of a Production Incident
Designing a Decomposition-and-QA-Pair Workflow for Contract Review with Recursive Escalation
Stabilizing a Decomposition-Based LLM Workflow for a Regulated Customer-Email Triage System