Learn Before
The Role of the Initial Token in Sequence Probability
A language model calculates the probability of a sequence of tokens by breaking it down into a series of conditional probabilities. Explain the specific, essential function of a special start symbol, such as [CLS], in initiating this calculation.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A researcher is preparing the sentence 'Language models are powerful.' for input into a specific type of model. This model architecture requires a special token, denoted as
[CLS], to be placed at the very beginning of every input sequence to mark its start. Which of the following options correctly formats the sentence for this model?The
[CLS]token, when used in a language model, is exclusively for aggregating sequence information for downstream classification tasks and does not serve as the initial token required to start the sequence's probability calculation.The Role of the Initial Token in Sequence Probability