Learn Before
Analyzing Model Reasoning Processes
A user provides a language model with the following word problem: 'A factory has 5 machines that can make 5 widgets in 5 minutes. At this rate, how long would it take 100 machines to make 100 widgets?'
Below are two different responses generated by the model based on two different prompting approaches:
Response A: 'It would take 100 minutes for 100 machines to make 100 widgets.'
Response B: 'Let's break this down. If 5 machines make 5 widgets in 5 minutes, this means each machine takes 5 minutes to make 1 widget. The machines work in parallel. Therefore, if you have 100 machines, each machine will still take 5 minutes to make its own widget. Since they all work at the same time, it will take 5 minutes to make 100 widgets.'
Analyze the two responses. Explain why Response B arrives at the correct answer while Response A does not, focusing specifically on the differences in the problem-solving process demonstrated in each output.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.3 Prompting - Foundations of Large Language Models
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Application of CoT to Algebraic Calculation Problems
A user wants a Large Language Model to solve a multi-step logic problem. They are considering two different prompts:
Prompt A: 'If a bat and a ball cost $1.10 in total, and the bat costs $1.00 more than the ball, how much does the ball cost?'
Prompt B: 'If a bat and a ball cost $1.10 in total, and the bat costs $1.00 more than the ball, how much does the ball cost? Let's think step by step.'
Which prompt is more likely to elicit a correct answer from the model, and what is the most accurate reason for its effectiveness?
Improving LLM Performance on Multi-Step Problems
Analyzing Model Reasoning Processes