The process of computing the log-probability of an output sequence given an input, $\log \Pr(\mathbf{y}|\mathbf{x})$, in a Transformer language model involves several sequential operations. First, the input $\mathbf{x}$ and output $\mathbf{y}$ are concatenated. For each position $i'$, the corresponding token embedding is processed through a stack of Transformer layers, where self-attention networks update the KV cache and compute attention outputs. If the position corresponds to a generated token, the model utilizes a Softmax layer to determine the token's prediction probability. Finally, the total log-probability is calculated by summing these individual token log-probabilities over the entire generated sequence.