Learn Before
Analysis of Sequence Order on Joint Probability
Consider two distinct, non-empty token sequences, x and y. A language model calculates the probability of the combined sequence [x, y] as Pr([x, y]) and the probability of the reverse combined sequence [y, x] as Pr([y, x]). Would a well-trained language model typically assign an equal probability to these two combined sequences (i.e., is Pr([x, y]) = Pr([y, x]) generally true)? Justify your answer based on how language models process sequential information.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Conditional vs. Joint Probability Objectives in Language Modeling
Relationship Between Joint, Conditional, and Marginal Log-Probabilities of Sequences
General Language Modeling Objective based on Joint Log-Probability
A language model is being used to determine the likelihood of a specific sentence. Let the input sequence
xbe 'The sun is' and the output sequenceybe 'shining brightly'. The notationPr([x, y])represents the probability of the model generating the full, combined sequence. Which statement best analyzes what this probability value signifies?Analysis of Sequence Order on Joint Probability
Conditional Log-Probability via Joint and Marginal Log-Probabilities
Model Comparison Using Joint Sequence Probability