Case Study

Calculating Reasoning Path Score

A language model generates a three-step reasoning path to solve a problem. A separate classification model evaluates each step, assigning a probability to potential labels. The final classification for a step, denoted as C(x,yˉk)C(\mathbf{x}, \bar{\mathbf{y}}_{\leq k}), is the label with the highest probability. The overall score for the path is calculated by summing up the number of steps classified as 'correct', based on the formula: r(x,y)=k=1nsδ(correct,C(x,yˉk))r(\mathbf{x}, \mathbf{y}) = \sum_{k=1}^{n_s} \delta(correct, C(\mathbf{x}, \bar{\mathbf{y}}_{\leq k})), where δ\delta is 1 if its two arguments are identical and 0 otherwise. Given the classifier's probability outputs below, what is the final score for the entire reasoning path?

0

1

Updated 2025-10-02

Contributors are:

Who are from:

Tags

Ch.5 Inference - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Application in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science