Learn Before
Interpreting Model Certainty from Ranked Probabilities
A language model is tasked with completing a sentence. In one instance (Case A), the ranking stage produces the following top three candidates: 'happy' (Pr=0.8), 'glad' (Pr=0.05), 'joyful' (Pr=0.02). In another instance (Case B), the ranking stage produces: 'run' (Pr=0.35), 'walk' (Pr=0.32), 'sprint' (Pr=0.30). Compare the distribution of probabilities after ranking in Case A versus Case B. What does this difference suggest about the model's certainty for its top choice in each case?
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model has generated a set of candidate tokens to follow a sequence, each with an assigned probability. Arrange these candidates in the correct descending order based on their probability, as would occur during the ranking stage of a selection process.
A language model generates the following five candidate tokens and their associated probabilities to complete a sentence: 'sky' (Pr=0.45), 'moon' (Pr=0.15), 'clouds' (Pr=0.35), 'stars' (Pr=0.04), 'blue' (Pr=0.01). What is the primary purpose of sorting these candidates by their probability in a process designed to select only the single most likely token?
Interpreting Model Certainty from Ranked Probabilities