Limitation of Question-Answer Pair Demonstrations in Few-Shot Prompting
Providing a Large Language Model with demonstrations consisting only of question-answer pairs can be insufficient for it to deduce the correct reasoning process. This limitation makes it difficult for the model to generalize and accurately solve similar problems that require multi-step thinking.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Example of a Mathematical Reasoning Task for LLMs
Constructing a Few-Shot Prompt for Multi-Step Reasoning
A developer is prompting a large language model to solve multi-step logic puzzles. They are comparing two few-shot prompting strategies. Strategy A provides examples showing only the puzzle and the final answer. Strategy B provides examples showing the puzzle, a step-by-step reasoning process, and then the final answer. Which strategy is more likely to yield consistently accurate results for new, complex puzzles, and why?
Limitation of Question-Answer Pair Demonstrations in Few-Shot Prompting
Diagnosing Prompting Failures in Multi-Step Tasks
Learn After
LLM's Answer (7) to the Prompt for Calculating the Average of 2, 4, and 9
Creating a CoT Prompt by Incorporating Reasoning Steps
Diagnosing a Flawed Prompting Strategy
A developer is trying to get a large language model to solve two-step arithmetic word problems. They use a few-shot prompting strategy, providing several examples. Each example consists of a word problem followed only by its final numerical answer (e.g., 'Problem: ... Answer: 15'). The model consistently fails to solve new, slightly different word problems. What is the most likely reason for the model's poor performance?
Diagnosing Prompt Insufficiency