1Cademy - Input Token Sequence in Language Models

Learn Before

Definition of a Prompt

Definition

Input Token Sequence in Language Models

In language modeling, an input sequence is mathematically defined as a sequence of tokens $x_0, x_1, \dots, x_m$ . The initial token, $x_0$ , functions as a special start symbol that marks the beginning of the sequence. This symbol is frequently denoted as $\langle s \rangle$ or $\langle \mathrm{SOS} \rangle$ , while architectures such as BERT represent it using the $[\mathrm{CLS}]$ token.

Updated 2026-04-18

Contributors are:

Who are from:

Learn After

Start of Sequence (SOS) Token
Formal Definition of LLM Inference
A user provides the input 'Summarize this article', which a language model processes into three distinct tokens ('Summarize', 'this', 'article'). Based on the formal structure where an input sequence is represented by its tokens plus a special start symbol, what is the total number of tokens in the complete sequence given to the model?
A language model receives an input prompt that is tokenized into 10 tokens. According to the formal representation of an input sequence, $\mathbf{x} = x_0...x_m$ , which of the following correctly describes the structure of the complete sequence processed by the model?
A language model is given the complete input token sequence: $\mathbf{x} = [⟨SOS⟩, \text{'What'}, \text{'is'}, \text{'the'}, \text{'capital'}, \text{'of'}, \text{'France'}, \text{'?'} ]$ . By analyzing the components of this sequence, identify which token's primary role is to signal the beginning of the input context for the model.

Learn Before

Related

Learn After