Learn Before
The Challenge of Multi-Step Logical Inference for LLMs in Arithmetic Reasoning
Solving mathematical word problems is a significant challenge for Large Language Models. The difficulty arises because these tasks demand more than simple problem-to-answer mapping; they require the model to perform multi-step logical inference. This involves translating natural language into a sequence of mathematical operations to arrive at the correct solution.
0
1
References
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.3 Prompting - Foundations of Large Language Models
Related
GSM8K Benchmark
Insufficiency of Simple Demonstrations for LLM Reasoning Tasks
A user gives a language model the following prompt: 'I have a box that contains a red ball and a blue ball. I take the red ball out and put it on the table. What is left in the box?' The model responds: 'The box contains a red ball and a blue ball.' Which of the following best analyzes the likely cause of the model's incorrect answer?
Commonsense Reasoning as a Challenging Task for LLMs
In-Context Learning (ICL)
The Challenge of Multi-Step Logical Inference for LLMs in Arithmetic Reasoning
Language Model Scheduling Error Analysis
Predicting LLM Reasoning Flaws
Learn After
Example of a Probability-Based Word Problem for LLMs
Example of a Multi-Step Arithmetic Word Problem (Swimming Pool)
Example of a Mathematical Reasoning Word Problem (Jessica's Apps)
Example of a Multi-Step Arithmetic Word Problem (Tom's Marbles)
A large language model was given the following word problem: 'A bakery had 20 muffins. They sold 12 muffins and then baked 3 dozen more. How many muffins does the bakery have now?' The model produced this response: 'First, we start with 20 muffins. They sold 12, so 20 - 12 = 8. Then they baked 3 more, so 8 + 3 = 11. The final answer is 11.' Which statement best analyzes the primary reasoning failure in the model's response?
Chain-of-Thought (COT) Prompting
Example of a Multi-Step Arithmetic Word Problem (Jack's Apples)
Evaluating LLM Arithmetic Inference
A language model is tasked with solving arithmetic word problems. Below are common types of errors it might make when translating language into a sequence of mathematical operations. Match each error type with the scenario that best exemplifies it.