A language model is being trained to predict the next word in a sentence. For the input context 'The sun is shining...', the ideal (target) probability distribution, denoted as $Pr^t$, gives a high probability to the word 'brightly'. The model's performance is measured by a loss function that compares the model's predicted probability distribution, $Pr_θ^s$, to the target distribution.

Consider two different sets of model parameters, θ₁ and θ₂:

• With parameters θ₁, the model's distribution $P

Interpreting a Model's Training Step

Comparing Model Performance via Loss