Example of a Multi-Step Arithmetic Word Problem (Jack's Apples)
This is a multi-step arithmetic word problem often used to test or demonstrate the reasoning capabilities of language models. The problem statement is: 'Jack has apples. He ate of them for dinner, but then his mom gave him more apples. The next day, Jack gave apples to his friend John. How many apples does Jack have left in the end?'
0
1
References
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Related
Example of a Probability-Based Word Problem for LLMs
Example of a Multi-Step Arithmetic Word Problem (Swimming Pool)
Example of a Mathematical Reasoning Word Problem (Jessica's Apps)
Example of a Multi-Step Arithmetic Word Problem (Tom's Marbles)
A large language model was given the following word problem: 'A bakery had 20 muffins. They sold 12 muffins and then baked 3 dozen more. How many muffins does the bakery have now?' The model produced this response: 'First, we start with 20 muffins. They sold 12, so 20 - 12 = 8. Then they baked 3 more, so 8 + 3 = 11. The final answer is 11.' Which statement best analyzes the primary reasoning failure in the model's response?
Chain-of-Thought (COT) Prompting
Example of a Multi-Step Arithmetic Word Problem (Jack's Apples)
Evaluating LLM Arithmetic Inference
A language model is tasked with solving arithmetic word problems. Below are common types of errors it might make when translating language into a sequence of mathematical operations. Match each error type with the scenario that best exemplifies it.
Learn After
Provided Answer (12) to the Example Arithmetic Reasoning Word Problem
Initial State for the Apple Problem
In-Context Learning (ICL)
A language model is presented with the following problem: "Jack has 7 apples. He ate 2 of them for dinner, but then his mom gave him 5 more apples. The next day, Jack gave 3 apples to his friend John. How many apples does Jack have left in the end?" The model processes the problem and performs the calculation
(7 + 5) - 2 - 3, arriving at the correct final answer of 7. Which of the following statements best analyzes the flaw in the model's problem-solving approach?A language model is tasked with solving the following word problem: 'Jack has 7 apples. He ate 2 of them for dinner, but then his mom gave him 5 more apples. The next day, Jack gave 3 apples to his friend John. How many apples does Jack have left in the end?' Arrange the following computational steps into the correct logical sequence that the model should follow to arrive at the final answer.
Analyzing a Flawed Arithmetic Reasoning Process
Incorrect Model Output () for the Jack's Apples Word Problem
Example of One-Shot Chain-of-Thought (COT) Prompting
Zero-Shot CoT Example with Jack's Apples