Learn Before
Token Classification in Discriminative Training
Token classification is a discriminative pre-training approach used for encoder-only models. In this method, the model receives an input sequence and performs a classification task for each individual token. For example, given the input [C] The kitten is chasing the ball ., the model outputs a supervision signal or probability distribution for every token, such as and .
0
1
Tags
Foundations of Large Language Models
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
A research team is pre-training a language model. They integrate this model as a component within a larger system designed to classify news articles into categories like 'Sports', 'Technology', or 'Politics'. The language model's internal parameters are updated based on the entire system's success in correctly categorizing the articles. Which statement best analyzes the fundamental principle of this training methodology?
Optimizing a Sentiment Analysis Model
Explaining a Pre-Training Supervision Method
Sentence Comparison in Discriminative Training
Token Classification in Discriminative Training