Formula

Incremental Calculation of Sequence Log-Probability

The log-probability of a generated sequence can be calculated incrementally at each step of the decoding process. For any sequence y=y1...yi\mathbf{y} = y_1...y_i, the total log-probability logPr(yx)\log \Pr(\mathbf{y}|\mathbf{x}) is evaluated as the sum of two components. The first component is the accumulated log-probability of the preceding sequence y<i\mathbf{y}_{<i}, representing the sum of the log-probabilities on the path from the root to the parent node that was computed in previous steps. The second component is the conditional token prediction log-probability of the current token yiy_i, which is newly computed by the large language model at the current step. The calculation is: logPr(yx)=logPr(y1...yix)=logPr(y<ix)accumulated up to the parent node+logPr(yix,y<i)newly computed for the current node\log \Pr(\mathbf{y}|\mathbf{x}) = \log \Pr(y_1...y_i|\mathbf{x}) = \underbrace{\log \Pr(\mathbf{y}_{<i}|\mathbf{x})}_{\text{accumulated up to the parent node}} + \underbrace{\log \Pr(y_i|\mathbf{x},\mathbf{y}_{<i})}_{\text{newly computed for the current node}}.

Image 0

0

1

Updated 2026-05-03

Contributors are:

Who are from:

Tags

Ch.5 Inference - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences