MLM Loss Function as Negative Log-Likelihood

A neural network is trained on a 4-class classification task. For a single training example where the true class is the second class, the model outputs the probability vector [0.1, 0.7, 0.1, 0.1]. The loss for this example is calculated as -log(0.7). This loss function can be interpreted as a measure of divergence between two probability distributions. What are these two distributions?

Interpreting Negative Log-Likelihood as Cross-Entropy

A neural network is being trained for a 3-class classification task (Classes A, B, C). For a single training example, the true label is 'Class B'. The model outputs the probability distribution P(A)=0.2, P(B)=0.5, P(C)=0.3. The loss for this example is calculated using the negative log-likelihood of the correct class, resulting in a loss of -log(0.5). This calculation is a direct application of the cross-entropy formula between the model's predicted distribution and the empirical distribution from the training data. What is the specific empirical probability distribution for this single training example?