Example of Auto-Regressive Probability Calculation
This example illustrates how an auto-regressive language model calculates the probability of a sentence like 'The kitten is chasing the ball .' by breaking it down into a sequence of conditional probabilities. The model predicts each word based on the words that precede it, represented by the series of calculations: Pr(路|The), Pr(路|kitten), and so on, until the entire sentence is processed.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Probability Factorization for Arbitrary Order Token Prediction
Step-by-Step Example of Auto-Regressive Sequence Generation
Standard Auto-Regressive Probability Factorization using Embeddings
A language model is designed to calculate the likelihood of a text sequence by predicting each token based only on the tokens that have come before it. Given the three-token sequence 'The quick brown', which of the following expressions correctly represents how this model would calculate the total probability of the entire sequence?
Example of Auto-Regressive Probability Calculation
Calculating Sequence Probability in an Auto-regressive Model
Debugging a Sequence Probability Calculation
Learn After
An auto-regressive language model calculates the probability of a sequence of words one step at a time, where each step depends on the words that came before it. Arrange the following calculations in the correct order that such a model would use to determine the total probability of the sentence 'The cat sat.'
An auto-regressive language model is tasked with calculating the total probability of the sentence 'The quick brown fox'. According to the principles of sequential probability calculation, which of the following expressions correctly represents the probability of the word 'brown' at its specific position in the sentence?
Sentence Probability Comparison