Learn Before
Sentence Comparison in Discriminative Training
Sentence comparison is a discriminative pre-training task applied to encoder-only models. In this task, the model processes two separate sentences and encodes them into representations, such as and . The model is then trained to output a score that compares or relates the two representations, formulated as .
0
1
Tags
Foundations of Large Language Models
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
A research team is pre-training a language model. They integrate this model as a component within a larger system designed to classify news articles into categories like 'Sports', 'Technology', or 'Politics'. The language model's internal parameters are updated based on the entire system's success in correctly categorizing the articles. Which statement best analyzes the fundamental principle of this training methodology?
Optimizing a Sentiment Analysis Model
Explaining a Pre-Training Supervision Method
Sentence Comparison in Discriminative Training
Token Classification in Discriminative Training