Learn Before
Probability of a Concatenated Token Sequence
The notation Pr([x, y]) represents the probability that a language model will generate the specific token sequence formed by concatenating an input sequence x and an output sequence y. This is also known as the joint probability of the sequences x and y.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Probability of a Concatenated Token Sequence
An input sequence of tokens is defined as
x = (The, cat, sat)and a subsequent output sequence is defined asy = (on, the, mat). Which of the following correctly represents the single, combined sequence denoted by[x, y]?Using [SEP] Tokens for Sequence Concatenation
Deconstructing a Concatenated Token Sequence
Given two token sequences,
x = (start, process)andy = (end, result), the concatenated sequence denoted by[x, y]is identical to the sequence denoted by[y, x].
Learn After
Conditional vs. Joint Probability Objectives in Language Modeling
Relationship Between Joint, Conditional, and Marginal Log-Probabilities of Sequences
General Language Modeling Objective based on Joint Log-Probability
A language model is being used to determine the likelihood of a specific sentence. Let the input sequence
xbe 'The sun is' and the output sequenceybe 'shining brightly'. The notationPr([x, y])represents the probability of the model generating the full, combined sequence. Which statement best analyzes what this probability value signifies?Analysis of Sequence Order on Joint Probability
Conditional Log-Probability via Joint and Marginal Log-Probabilities
Model Comparison Using Joint Sequence Probability