Calculating a Reasoning Path Score
An AI model generates a 5-step reasoning path to solve a problem. A separate classification model evaluates each step and produces the following labels: [Correct, Correct, Incorrect, Correct, Incorrect]. Based on the method of scoring by counting the number of correct steps, what is the final score for this reasoning path? Briefly explain how you arrived at this score.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formula for Scoring Reasoning Paths by Counting Correct Steps
Scoring an AI's Reasoning Process
An AI's multi-step solution to a complex problem is evaluated by a separate model that classifies each step as either 'correct' or 'incorrect'. The final quality score for the entire solution is calculated by summing the total number of steps classified as 'correct'. What is a primary conceptual limitation of this evaluation approach?
Calculating a Reasoning Path Score