Calculating Reasoning Path Score
A language model generates a three-step reasoning path to solve a problem. A separate classification model evaluates each step, assigning a probability to potential labels. The final classification for a step, denoted as , is the label with the highest probability. The overall score for the path is calculated by summing up the number of steps classified as 'correct', based on the formula: , where is 1 if its two arguments are identical and 0 otherwise. Given the classifier's probability outputs below, what is the final score for the entire reasoning path?
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Calculating Reasoning Path Score
Consider two reasoning paths, Path A and Path B, generated to solve the same problem. Path A consists of 3 steps, all of which are classified as 'correct'. Path B consists of 5 steps, where 4 are classified as 'correct' and 1 is 'incorrect'. According to the scoring formula , which of the following statements is true?
Critique of a Reasoning Path Scoring Method