Evaluating AI Reasoning in a Probability Problem
An AI model is given a probability word problem and produces the response below. Although the final answer is correct, evaluate the reliability of the model's reasoning process. Explain why this process might be considered weak or unreliable for solving similar, more complex problems.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Calculating the Probability of Exactly One Head in Three Coin Flips
An AI model is prompted with the following word problem: 'Three friends play a game where they flip a fair coin. Each friend flips the coin once. What is the probability that exactly one of them flips heads?'
Below are two different initial responses generated by the AI. Analyze the strategies and determine which response represents a more effective approach for solving this type of multi-step reasoning problem.
Response A: "The probability of one person getting heads is 1/2. Since there a
Evaluating AI Reasoning in a Probability Problem
LLM Reasoning on Probability Word Problems