Learn Before
Start of Sequence (SOS) Token
The (Start of Sequence) token, or , is a special symbol used to mark the beginning of an input sequence in a Large Language Model. It is often denoted as the first token, , in the sequence.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.2 Generative Models - Foundations of Large Language Models
Related
Start of Sequence (SOS) Token
Formal Definition of LLM Inference
A user provides the input 'Summarize this article', which a language model processes into three distinct tokens ('Summarize', 'this', 'article'). Based on the formal structure where an input sequence is represented by its tokens plus a special start symbol, what is the total number of tokens in the complete sequence given to the model?
A language model receives an input prompt that is tokenized into 10 tokens. According to the formal representation of an input sequence, , which of the following correctly describes the structure of the complete sequence processed by the model?
A language model is given the complete input token sequence: . By analyzing the components of this sequence, identify which token's primary role is to signal the beginning of the input context for the model.
Learn After
A specific language model requires that any input sequence must begin with a special symbol, denoted as
[START], to signal the beginning of the text. Given the sentence 'The model predicts.', which of the following token sequences is correctly formatted for this model?Debugging Model Input
Evaluating Model Design Choices