Learn Before
Predicting LLM Reasoning Flaws
A store has 20 boxes of apples. They sell 8 boxes in the morning. In the afternoon, they receive a new shipment of 12 boxes. However, they discover that 3 of the boxes from the new shipment are spoiled and must be discarded. Based on the known challenges language models face with multi-step reasoning, describe one plausible but incorrect final answer the model might generate for the total number of good boxes, and explain the flawed step-by-step process that would lead to that answer.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.3 Prompting - Foundations of Large Language Models
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
GSM8K Benchmark
Insufficiency of Simple Demonstrations for LLM Reasoning Tasks
A user gives a language model the following prompt: 'I have a box that contains a red ball and a blue ball. I take the red ball out and put it on the table. What is left in the box?' The model responds: 'The box contains a red ball and a blue ball.' Which of the following best analyzes the likely cause of the model's incorrect answer?
Commonsense Reasoning as a Challenging Task for LLMs
In-Context Learning (ICL)
The Challenge of Multi-Step Logical Inference for LLMs in Arithmetic Reasoning
Language Model Scheduling Error Analysis
Predicting LLM Reasoning Flaws