Designing a Robust Prompting Workflow for Multi-Step Root-Cause Analysis with Limited Examples
You are rolling out an internal LLM assistant for SREs to perform root-cause analysis (RCA) from incident artifacts. The assistant receives: (1) a short incident timeline, (2) 30–60 lines of logs, and (3) a list of recent deploys. The desired output is a single RCA summary with: likely cause, supporting evidence, and a recommended next action.
Constraints: You can only include ONE short demonstration in the prompt due to context limits; the assistant must work on novel incidents; and the first pilot showed two failure modes: (a) the model jumps to a plausible cause without checking evidence across artifacts, and (b) when you add “Let’s think step by step,” it sometimes produces a long analysis but forgets to state a clear final RCA.
As the prompt/workflow designer, propose a concrete prompting plan that combines: (i) in-context learning with the single demonstration, (ii) explicit problem decomposition (including a least-to-most style progression of sub-questions), (iii) chain-of-thought elicitation (including how you would use or avoid zero-shot CoT triggers), and (iv) self-refinement (critique + revision) to reduce both failure modes. Your answer must include the exact structure of the prompt(s) you would send (section headings and what each section contains) and explain why each component is placed where it is.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.3 Prompting - Foundations of Large Language Models
Ch.5 Inference - Foundations of Large Language Models
Ch.1 Pre-training - Foundations of Large Language Models
Related
Application of COT Prompting on GSM8K Benchmark
Structuring Logical Reasoning Steps for Demonstrations
Zero-Shot Chain-of-Thought (COT) Prompting
Application of CoT to Algebraic Calculation Problems
Benefits of Chain-of-Thought (CoT) Prompting
Incomplete Answers from Zero-Shot CoT Prompts
Chain-of-Thought as a Search Process
Supervising Intermediate Reasoning Steps for LLM Alignment
Limitations of Simple Chain-of-Thought Prompting
Creating a CoT Prompt by Incorporating Reasoning Steps
Alternative Trigger Phrases for Zero-Shot CoT Prompting
Incomplete Answers as a Potential Issue in Zero-Shot CoT Prompting
A developer is trying to improve a language model's ability to solve multi-step word problems. They compare two prompting strategies.
Strategy 1: Provide the model with a new word problem and ask for the final answer directly.
Strategy 2: Provide the model with a new word problem, but first show it an example of a similar problem where the solution is explicitly broken down into logical, sequential steps before reaching the final conclusion.
Why is Strategy 2 generally more effective
Improving a Prompt for a Multi-Step Problem
Few-Shot Chain-of-Thought (CoT) Prompting
Practical Limitations of Chain-of-Thought Prompting
The primary benefit of a prompting technique that demonstrates a step-by-step reasoning process is that it permanently modifies the language model's internal weights, making it inherently better at solving similar problems in the future, even without the detailed prompt.
Designing a Prompting Workflow for a High-Stakes, Multi-Step Task
Choosing and Justifying a Prompting Strategy Under Context and Quality Constraints
Diagnosing and Redesigning a Prompting Approach for a Decomposed Workflow
Stabilizing an LLM Workflow for Multi-Step Policy Compliance Decisions
Debugging a Multi-Step LLM Workflow for Contract Clause Risk Triage
Designing a Robust Prompting Workflow for Multi-Step Root-Cause Analysis with Limited Examples
You’re building an internal LLM assistant to help ...
Your team is rolling out an internal LLM assistant...
You’re leading an internal enablement team buildin...
You’re building an internal LLM workflow to produc...
Example of One-Shot Chain-of-Thought (COT) Prompting
Problem-Solving Scenarios for Chain-of-Thought Prompting
Self-Consistency Method
Sub-problem Generation in Least-to-Most Prompting
Improving Least-to-Most Prompting with Advanced Techniques
Improving Problem Decomposition in Least-to-Most Prompting
An AI developer needs a large language model to solve a complex, multi-step logic puzzle that requires deducing a final answer from a series of interdependent clues. Initial attempts to solve the puzzle by providing the full puzzle and a few examples of other solved puzzles have consistently failed. Which of the following prompting strategies is the most effective next step, and why?
Analyzing a Problem-Solving Approach
A language model is tasked with solving the following logic puzzle: 'Sarah, David, and Emily are a doctor, a lawyer, and an engineer. The doctor is Emily's sister. David is not the lawyer.' To solve this complex problem, it is broken down into a series of simpler, sequential sub-problems. Arrange the following sub-problems in the correct logical order that builds towards the final solution.
Your team is rolling out an internal LLM assistant...
You’re building an internal LLM workflow to produc...
You’re building an internal LLM assistant to help ...
You’re leading an internal enablement team buildin...
Choosing and Justifying a Prompting Strategy Under Context and Quality Constraints
Designing a Prompting Workflow for a High-Stakes, Multi-Step Task
Diagnosing and Redesigning a Prompting Approach for a Decomposed Workflow
Stabilizing an LLM Workflow for Multi-Step Policy Compliance Decisions
Debugging a Multi-Step LLM Workflow for Contract Clause Risk Triage
Designing a Robust Prompting Workflow for Multi-Step Root-Cause Analysis with Limited Examples
Example of Final Problem Solving in Least-to-Most Prompting
Example of Self-Refinement in Machine Translation
Three-Step Framework for Self-Refinement in LLMs
Ideal Self-Refinement without Additional Training