A language model calculates the total log probability for two different sequences of words. The total log probability for Sequence A is -8.7, and the total log probability for Sequence B is -10.2. Based solely on these values, what can be concluded about the likelihood of these two sequences?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is tasked with calculating the joint probability of a very long sequence of words, such as an entire book chapter. The model computes the conditional probability for each word given its preceding context. When the model attempts to find the total probability of the chapter by multiplying these thousands of individual conditional probabilities (which are all fractions less than 1), which computational issue is most likely to occur, and why is converting the calculation to a sum of logarithms the standard solution?
Calculating Sequence Log Probability
A language model calculates the total log probability for two different sequences of words. The total log probability for Sequence A is -8.7, and the total log probability for Sequence B is -10.2. Based solely on these values, what can be concluded about the likelihood of these two sequences?