Example of a Multi-Step Arithmetic Word Problem (Tom's Marbles)
An illustrative multi-step arithmetic word problem involves Tom, who starts with marbles. He then wins , loses , and is given another . The question is to determine the final number of marbles Tom possesses.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Related
Example of a Probability-Based Word Problem for LLMs
Example of a Multi-Step Arithmetic Word Problem (Swimming Pool)
Example of a Mathematical Reasoning Word Problem (Jessica's Apps)
Example of a Multi-Step Arithmetic Word Problem (Tom's Marbles)
A large language model was given the following word problem: 'A bakery had 20 muffins. They sold 12 muffins and then baked 3 dozen more. How many muffins does the bakery have now?' The model produced this response: 'First, we start with 20 muffins. They sold 12, so 20 - 12 = 8. Then they baked 3 more, so 8 + 3 = 11. The final answer is 11.' Which statement best analyzes the primary reasoning failure in the model's response?
Chain-of-Thought (COT) Prompting
Example of a Multi-Step Arithmetic Word Problem (Jack's Apples)
Evaluating LLM Arithmetic Inference
A language model is tasked with solving arithmetic word problems. Below are common types of errors it might make when translating language into a sequence of mathematical operations. Match each error type with the scenario that best exemplifies it.
One-Shot Chain-of-Thought (COT) Prompting
Improving Model Guidance for Multi-Step Problems
Example of a Multi-Step Arithmetic Word Problem (Tom's Marbles)
You are creating a demonstration to teach a large language model how to solve multi-step arithmetic word problems. The goal is to show the model a clear, step-by-step reasoning process that is easy to follow.
Problem: A library starts the day with 80 books on the 'new arrivals' shelf. During the day, 45 books are checked out. Later, librarians add a new shipment of 22 books to the shelf. How many books are on the shelf at the end of the day?
Which of the following demonstrations is structured most effectively to guide the model?
You are preparing a demonstration to show a language model how to solve a word problem. Arrange the following reasoning steps into the most logical and clear sequence to arrive at the correct answer for the problem below.
Problem: A bakery starts with 20 bags of flour. They use 5 bags for bread and 3 bags for cakes. Later in the day, they receive a delivery of 10 more bags. How many bags of flour does the bakery have at the end of the day?
Learn After
First Reasoning Step for the ICL Demonstration on Marbles
A language model is tasked with solving the following word problem: 'Tom starts with 12 marbles. He then wins 7, loses 5, and is given another 3. How many marbles does Tom have in the end?' To guide the model's reasoning process, which of the following sequences of steps most accurately and logically breaks down the problem into a step-by-step calculation that follows the narrative?
Analyzing a Language Model's Reasoning Process
A language model is asked to solve the following word problem: 'Tom starts with 12 marbles. He then wins 7, loses 5, and is given another 3. How many marbles does Tom have in the end?' The model produces the reasoning below. Identify the step where the model makes a logical error in translating the problem's narrative into a mathematical operation.
Step 1: Tom begins with 12 marbles. Step 2: Tom wins 7 marbles, so the total becomes 12 + 7 = 19. Step 3: Tom loses 5 marbles, so the total becomes 19 + 5 = 24. Step 4: Tom is given 3 marbles, so the final total is 24 + 3 = 27.