Learn Before
Evaluating Sequence Likelihood with Log Probabilities
You are evaluating a language model that has generated two possible three-word sequences. Instead of raw probabilities, the model outputs the log probability for each word given its context. Based on the values below, determine which sequence is more likely according to the model and state its total log probability.
0
1
Tags
Data Science
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is designed to calculate the probability of a long sentence by sequentially multiplying the conditional probabilities of each word. Each individual word probability is a small floating-point number (e.g., 0.05, 0.1, 0.02). During testing on sentences with over 100 words, the model consistently outputs a final probability of 0.0, even though no single word has a probability of zero. What is the most likely technical reason for this incorrect result?
Comparing Sequence Probabilities in Log Space
Evaluating Sequence Likelihood with Log Probabilities