Evaluating and Redesigning a Decomposition Workflow Under Context and Cost Constraints
You are leading an internal enablement team building an LLM-powered assistant to answer employee questions about a 120-page, frequently updated Benefits & HR policy PDF. The assistant must answer multi-hop questions (e.g., “If I switch from full-time to part-time mid-year, how does that affect my HSA eligibility and the employer match, and what forms do I need?”). You have a strict constraint: each user question can trigger at most 6 LLM calls, and the total context window is limited, so you cannot paste the entire policy or all prior intermediate work into every call.
A junior engineer proposes this workflow:
- Generate a fixed list of 10 sub-questions up front for any user query.
- Solve them in order.
- For each step, only pass the immediately previous sub-question and answer (not the original user question or earlier Q/A pairs).
- If a sub-question is hard, “just ask the model to think harder,” without further decomposition.
Write an evaluation of this proposed workflow and then propose a revised workflow that would be more reliable under the constraints. Your answer must (a) explain how you would generate sub-problems so they are neither too generic nor too rigid, (b) describe when and how you would apply recursive decomposition to a sub-problem that is still too complex, and (c) specify exactly what contextual information you would carry forward between sequential steps (e.g., which Q/A pairs, what summaries, what citations) and why that choice improves correctness while respecting the context-window and 6-call limits. Conclude by describing one concrete failure mode your redesign prevents and the mechanism by which it prevents it.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Divide-and-Conquer Paradigm
Example of a Classification Task for LLMs: Identifying AI Risks in a Document
Approaches to Multi-Step Reasoning in LLMs
Two-Step Problem Decomposition
Dynamic Problem Decomposition for Complex Reasoning
Compositionality in NLP
Outlining as a Method of Problem Decomposition for Generative Tasks
General Framework of Problem Decomposition
A team is using a large language model to automate complex tasks. They decide to implement a strategy where a main problem is broken down into a complete, fixed list of sub-problems before the model begins to solve any of them. The model will then solve each sub-problem in sequence. For which of the following tasks is this pre-defined decomposition approach LEAST likely to succeed?
Evaluating a Problem Decomposition Strategy for Multi-Hop QA
Illustrating the Need for Decomposition in Generative Tasks
Complex Reasoning Problems
Multi-hop Question Answering
A development team is building several applications powered by a large language model. Match each application's primary task with the most suitable strategy for breaking down the problem.
Designing a Decomposition-Driven LLM Workflow for a High-Stakes Corporate Task
Debugging a Decomposition-Based LLM Workflow Using Recursive Sub-Problems and Contextual QA Pairs
Evaluating and Redesigning a Decomposition Workflow Under Context and Cost Constraints
Designing a Decomposition-and-QA-Pair Workflow for Contract Review with Recursive Escalation
Stabilizing a Decomposition-Based LLM Workflow for a Regulated Customer-Email Triage System
Designing a Decomposition Workflow for Root-Cause Analysis of a Production Incident
Create a Recursive, Context-Carrying Decomposition Plan for LLM-Assisted KPI Narrative Generation
You are building an internal LLM assistant to answ...
You are designing an internal LLM workflow to answ...
You’re building an internal LLM workflow to answer...
Your team is rolling out an internal LLM assistant...
You’re building an internal LLM workflow to produc...
You’re building an internal LLM assistant to help ...
You’re leading an internal enablement team buildin...
Choosing and Justifying a Prompting Strategy Under Context and Quality Constraints
Designing a Prompting Workflow for a High-Stakes, Multi-Step Task
Diagnosing and Redesigning a Prompting Approach for a Decomposed Workflow
Stabilizing an LLM Workflow for Multi-Step Policy Compliance Decisions
Debugging a Multi-Step LLM Workflow for Contract Clause Risk Triage
Designing a Robust Prompting Workflow for Multi-Step Root-Cause Analysis with Limited Examples
Psychological Perspective on Problem Decomposition
Tool Use as Problem Decomposition in LLMs
Decomposing a Complex Planning Task
A language model is tasked with generating a comprehensive report on the impact of remote work on employee productivity and well-being. Which of the following sets of sub-problems represents the most effective initial breakdown of this complex task to ensure a well-structured and complete final output?
Evaluating a Generated Problem Breakdown
You are building an internal LLM assistant to answ...
You are designing an internal LLM workflow to answ...
You’re building an internal LLM workflow to answer...
Create a Recursive, Context-Carrying Decomposition Plan for LLM-Assisted KPI Narrative Generation
Designing a Decomposition-Driven LLM Workflow for a High-Stakes Corporate Task
Evaluating and Redesigning a Decomposition Workflow Under Context and Cost Constraints
Debugging a Decomposition-Based LLM Workflow Using Recursive Sub-Problems and Contextual QA Pairs
Designing a Decomposition Workflow for Root-Cause Analysis of a Production Incident
Designing a Decomposition-and-QA-Pair Workflow for Contract Review with Recursive Escalation
Stabilizing a Decomposition-Based LLM Workflow for a Regulated Customer-Email Triage System
Dynamic Sub-Problem Generation
Sequential Sub-Problem Solving with Contextual QA Pairs
Expanding the Sub-Problem Solver Beyond LLMs
Recursive Decomposition of Sub-Problems
Framing Problem-Solving as a Reinforcement Learning Problem
An AI is tasked with creating a valid three-day weekend itinerary (Fri, Sat, Sun) to visit a museum, a park, and a specific restaurant. The AI first decomposes the problem and solves two sub-problems, yielding the following intermediate conclusions:
- The museum is only open on Friday and Saturday.
- The restaurant requires a reservation made at least one day in advance.
Which of the following statements best describes the next step in the sub-problem solving process to generate the final itinerary?
Synthesizing Sub-Problem Solutions
Analyzing a Flawed Project Plan
You are building an internal LLM assistant to answ...
You are designing an internal LLM workflow to answ...
You’re building an internal LLM workflow to answer...
Create a Recursive, Context-Carrying Decomposition Plan for LLM-Assisted KPI Narrative Generation
Designing a Decomposition-Driven LLM Workflow for a High-Stakes Corporate Task
Evaluating and Redesigning a Decomposition Workflow Under Context and Cost Constraints
Debugging a Decomposition-Based LLM Workflow Using Recursive Sub-Problems and Contextual QA Pairs
Designing a Decomposition Workflow for Root-Cause Analysis of a Production Incident
Designing a Decomposition-and-QA-Pair Workflow for Contract Review with Recursive Escalation
Stabilizing a Decomposition-Based LLM Workflow for a Regulated Customer-Email Triage System
Hierarchical Problem-Solving Structures
AI Task Management Strategy
An AI system is designed to plan a complex software project. It first breaks the project into three main tasks: 'Design Architecture', 'Develop Modules', and 'Test System'. The system successfully completes 'Design Architecture'. However, when it attempts 'Develop Modules', it gets stuck because this task itself consists of many interdependent sub-tasks like 'Develop UI', 'Develop Backend API', and 'Set up Database', each with its own complexities. What is the fundamental flaw in the system's problem-solving process?
An AI system is tasked with solving a complex problem. Arrange the following steps in the logical order that demonstrates a recursive approach to breaking down a sub-problem.
You are building an internal LLM assistant to answ...
You are designing an internal LLM workflow to answ...
You’re building an internal LLM workflow to answer...
Create a Recursive, Context-Carrying Decomposition Plan for LLM-Assisted KPI Narrative Generation
Designing a Decomposition-Driven LLM Workflow for a High-Stakes Corporate Task
Evaluating and Redesigning a Decomposition Workflow Under Context and Cost Constraints
Debugging a Decomposition-Based LLM Workflow Using Recursive Sub-Problems and Contextual QA Pairs
Designing a Decomposition Workflow for Root-Cause Analysis of a Production Incident
Designing a Decomposition-and-QA-Pair Workflow for Contract Review with Recursive Escalation
Stabilizing a Decomposition-Based LLM Workflow for a Regulated Customer-Email Triage System
Synthesizing Sub-Problem Answers for a Final Solution
Formula for Sequential Sub-Problem Solving
Prompting an LLM for the First Sub-Problem in a Sequence
Example of Constructing a Contextual Prompt for a Subsequent Sub-Problem
An AI assistant is tasked with planning a multi-day hiking trip. The task is broken down into a sequence of three steps. To solve the third and final step, 'Create a detailed meal plan,' two different approaches for providing the model with information are proposed.
- Approach X: The model is given the original task, the first question ('What is the weather forecast?') and its answer, the second question ('What gear is needed?') and its answer, and the final question ('Create a detailed meal plan.').
- Approach Y: The model is given only the second question ('What gear is needed?') and its answer, along with the final question ('Create a detailed meal plan.').
Which approach is more likely to produce a high-quality, contextually appropriate final answer, and why?
Constructing a Contextual Prompt for a Subsequent Sub-Problem
Synthesizing Sub-Problem Answers to Solve the Original Problem
An AI assistant is solving a complex problem by breaking it down into smaller, sequential questions. You need to construct the complete input that should be sent to the model to solve the third sub-problem. Arrange the following components in the correct logical order to create this input.
Diagnosing a Flawed AI Problem-Solving Process
You are building an internal LLM assistant to answ...
You are designing an internal LLM workflow to answ...
You’re building an internal LLM workflow to answer...
Create a Recursive, Context-Carrying Decomposition Plan for LLM-Assisted KPI Narrative Generation
Designing a Decomposition-Driven LLM Workflow for a High-Stakes Corporate Task
Evaluating and Redesigning a Decomposition Workflow Under Context and Cost Constraints
Debugging a Decomposition-Based LLM Workflow Using Recursive Sub-Problems and Contextual QA Pairs
Designing a Decomposition Workflow for Root-Cause Analysis of a Production Incident
Designing a Decomposition-and-QA-Pair Workflow for Contract Review with Recursive Escalation
Stabilizing a Decomposition-Based LLM Workflow for a Regulated Customer-Email Triage System