Learn Before
Evaluating Sentence Plausibility
A language model is tasked with completing the prompt: "The cat sat on the...". It has generated two possible completions and calculated the conditional log-probabilities for each token in the sequence. Based on these values, calculate the total log-likelihood for each sentence and determine which one the model considers more plausible. Justify your conclusion.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.5 Inference - Foundations of Large Language Models
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Log-Probability of a Ranked Sequence
Log-Likelihood Objective for Language Model Training
A language model is generating a sequence of tokens. It has computed the following conditional log-probabilities for a three-token sequence, where each token's probability is dependent on the ones that came before it:
- Log-probability of the first token: -1.8
- Log-probability of the second token, given the first: -2.5
- Log-probability of the third token, given the first two: -1.2
Based on these values, what is the total log-likelihood of this entire three-token sequence?
Evaluating Sentence Plausibility
A language model has calculated the total log-likelihood for the sequence of tokens: ["The", "quick", "brown", "fox"]. The calculation involves summing the conditional log-probabilities of each token given the preceding ones. If the third token is changed from "brown" to "lazy", creating the new sequence ["The", "quick", "lazy", "fox"], which set of conditional log-probabilities must be re-calculated to find the new total log-likelihood?
Applying Log-Likelihood Calculation to a Training Dataset