Calculating Sequence Log Probability
A language model is evaluating the three-word sequence 'the cat sat'. It has computed the following conditional log probabilities:
log Pr('the') = -1.2log Pr('cat' | 'the') = -2.5log Pr('sat' | 'the', 'cat') = -1.8
Based on the principle that the log probability of a sequence is the sum of the log probabilities of its components, what is the total log probability of the entire sequence 'the cat sat'?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is tasked with calculating the joint probability of a very long sequence of words, such as an entire book chapter. The model computes the conditional probability for each word given its preceding context. When the model attempts to find the total probability of the chapter by multiplying these thousands of individual conditional probabilities (which are all fractions less than 1), which computational issue is most likely to occur, and why is converting the calculation to a sum of logarithms the standard solution?
Calculating Sequence Log Probability
A language model calculates the total log probability for two different sequences of words. The total log probability for Sequence A is -8.7, and the total log probability for Sequence B is -10.2. Based solely on these values, what can be concluded about the likelihood of these two sequences?