Next Sentence Prediction Loss Formula

A language model is being trained on a binary classification task to determine if two sentences are consecutive. The model's performance is optimized by minimizing a loss value derived from a special token that aggregates information about the sentence pair. If the loss value for this task consistently decreases during training, what is the most accurate interpretation of the model's learning progress?

Diagnosing Training Failure in a Sentence Relationship Task

During the training of a language model on the task of predicting sentence relationships, if the classifier component assigns a very high probability to the correct relationship label for a given sentence pair, the corresponding loss value calculated for that pair will also be very high.