Learn Before
Formula

Conditional Probability of the Next Element in a Sequence

The formula Prθ^(xi+1x0,...,xi)Pr_{\hat{\theta}}(x_{i+1}|x_0, ..., x_i) represents the conditional probability of the next element in a sequence, xi+1x_{i+1}, given all the preceding elements, x0,...,xix_0, ..., x_i. The subscript θ^\hat{\theta} indicates that this probability is estimated by a model with parameters θ^\hat{\theta}. This is a fundamental concept in autoregressive models, such as language models, which generate sequences one element at a time based on the history of previously generated elements.

Image 0

0

1

Updated 2025-10-08

Contributors are:

Who are from:

Tags

Ch.1 Pre-training - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Related