Analyzing a Model's Prediction Choice
A language model is tasked with completing a sentence. For a given input context, it calculates the following probabilities for the next word: Pr('mat'|context) = 0.4, Pr('chair'|context) = 0.3, Pr('roof'|context) = 0.2, and Pr('dog'|context) = 0.1. Based on the principle of selecting the output with the maximum probability, identify the model's prediction. Then, explain why a model strictly following this principle would not choose 'chair', even though it is a plausible real-world completion.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.4 Alignment - Foundations of Large Language Models
Ch.5 Inference - Foundations of Large Language Models
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
LLM Prediction with Full Context
LLM Prediction with Compressed Context
Mathematical Formulation of Prompt Ensembling
Formula for Scoring Reasoning Paths by Counting Correct Steps
A classification model is given an input,
x, and must choose an output,y, from the set of possible classes {A, B, C, D}. The model's decision rule is to select the class that has the highest conditional probability,Pr(y|x). Given the following probabilities calculated by the model for the inputx, what will its final prediction be?Pr(y=A | x)= 0.15Pr(y=B | x)= 0.55Pr(y=C | x)= 0.25Pr(y=D | x)= 0.05
Model Prediction vs. Ground Truth
Analyzing a Model's Prediction Choice