Concept

BERT Input Format for Sentence Pairs

When handling sentence pairs, BERT processes them as a unified sequence. This sequence begins with a [CLS] token, followed by the first sentence (denoted as SentA\mathrm{Sent}_{A}), a separator token [SEP], the second sentence (SentB\mathrm{Sent}_{B}), and a concluding [SEP] token. As established in the original BERT paper, the [SEP] token explicitly marks the boundary between the two sentences. This general input representation is formally expressed as the sequence: [CLS] SentA [SEP] SentB [SEP].

0

1

Updated 2026-05-02

Contributors are:

Who are from:

Tags

Ch.1 Pre-training - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Ch.2 Generative Models - Foundations of Large Language Models

Related