Learn Before
Evaluating Prompting Strategies for Compounding Errors
Consider two different multi-step approaches for generating a detailed analysis of a lengthy research paper using a language model.
-
Approach A: The model first generates a summary of Chapter 1. Then, it uses that summary as the sole context to generate a summary of Chapter 2. This process continues, with the summary of Chapter N being used as the context for summarizing Chapter N+1.
-
Approach B: For each chapter, the model is given the original full text of the research paper along with the summary of the previous chapter to generate the summary for the current chapter.
Which approach is more vulnerable to a compounding negative impact on the final outcome if a mistake is made in an early step? Justify your answer.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Analyzing a Flawed Multi-Step Prompting Process
A user is generating a complex, multi-part technical document using a language model. The process involves generating the first part, creating a summary of it, and then using that summary as context to generate the second part. This continues for all subsequent parts. In the summary of the second part, a key technical specification is accidentally inverted (e.g., 'minimum tolerance' is written as 'maximum tolerance'). The user does not catch this error and continues the process. As a result, the final parts of the document are incoherent and contain conclusions that are technically unsound. Which of the following statements best explains the root cause of the final document's failure?
Evaluating Prompting Strategies for Compounding Errors