Evaluating a Language Model's Reasoning Error
A language model was given the problem: "Jack has 7 apples. He ate 2, his mom gave him 5 more, and then he gave 3 to a friend. How many apples are left?" The model answered "12". Evaluate the severity of this error. Is it a simple calculation mistake or does it indicate a more fundamental flaw in the model's reasoning process? Justify your answer.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Analysis of an Arithmetic Reasoning Output
A language model is presented with the following scenario: "Jack has 7 apples. He ate 2 of them, then his mom gave him 5 more. The next day, Jack gave 3 apples to his friend." The model is asked how many apples Jack has left and responds with "12". Which of the following statements best analyzes the model's response?
Evaluating a Language Model's Reasoning Error