Learn Before
A language model is being trained on a binary classification task to determine if two sentences are consecutive. The model's performance is optimized by minimizing a loss value derived from a special token that aggregates information about the sentence pair. If the loss value for this task consistently decreases during training, what is the most accurate interpretation of the model's learning progress?
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Next Sentence Prediction Loss Formula
A language model is being trained on a binary classification task to determine if two sentences are consecutive. The model's performance is optimized by minimizing a loss value derived from a special token that aggregates information about the sentence pair. If the loss value for this task consistently decreases during training, what is the most accurate interpretation of the model's learning progress?
Diagnosing Training Failure in a Sentence Relationship Task
During the training of a language model on the task of predicting sentence relationships, if the classifier component assigns a very high probability to the correct relationship label for a given sentence pair, the corresponding loss value calculated for that pair will also be very high.