LLM Prediction with Full Context

LLM Prediction with Compressed Context

Mathematical Formulation of Prompt Ensembling

Formula for Scoring Reasoning Paths by Counting Correct Steps

A classification model is given an input, x, and must choose an output, y, from the set of possible classes {A, B, C, D}. The model's decision rule is to select the class that has the highest conditional probability, Pr(y|x). Given the following probabilities calculated by the model for the input x, what will its final prediction be?

• Pr(y=A | x) = 0.15
• Pr(y=B | x) = 0.55
• Pr(y=C | x) = 0.25
• Pr(y=D | x) = 0.05

Model Prediction vs. Ground Truth

Analyzing a Model's Prediction Choice