Calculating Joint Log-Probability
A language model is evaluating a potential completion for a given prompt. The model determines that the conditional log-probability of the completion (y) given the prompt (x) is -4.5. It also calculates the marginal log-probability of the prompt (x) itself to be -9.2. Based on this information, what is the joint log-probability of the combined sequence [x, y]?
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is given an input sequence, x. The model calculates the log-probability of this input sequence as -12.5. The model then generates a completion, y, and calculates the log-probability of the full, combined sequence [x, y] as -15.0. Based on these values, what is the conditional log-probability of the model generating the completion y given the input x?
Comparing Language Model Outputs
Calculating Joint Log-Probability