Learn Before
Differentiating Encoder and Decoder Sequences with Start Symbols
While it is possible to use a single, identical start symbol for both encoder and decoder input sequences, a common practice is to employ distinct symbols. This approach helps to clearly differentiate between the source sequence fed into the encoder and the target sequence processed by the decoder.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Related
Example of a T5 Machine Translation Training Sample with Special Tokens
Example of a T5 Question-Answering Sample
Example of a T5 Simplification Task Sample
Differentiating Encoder and Decoder Sequences with Start Symbols
Versatility of the T5 Text-to-Text Format
Definition of c_gold
Formula for Input Embedding Composition
A researcher wants to train a model to perform a new task: converting a sentence from passive voice to active voice. Given the passive input sentence 'The cake was eaten by the dog' and the desired active output 'The dog ate the cake', which of the following training samples is correctly structured according to the unified, prefix-based text-to-text format?
Critiquing a Text-to-Text Training Sample
A single text-to-text model is being trained on a dataset containing samples for four different tasks. Each sample's input begins with a prefix that instructs the model on what to do. Match each input sample (Source Text) with the most likely task it is intended for.
Learn After
Debugging Input Representation in a Sequence-to-Sequence Model
Example of a T5 Machine Translation Training Sample with Special Tokens
In designing a sequence-to-sequence model, an engineer decides to use one specific start symbol for all source sequences fed to the encoder and a different, unique start symbol for all target sequences fed to the decoder. Which statement best analyzes the primary benefit of this design choice?
In a sequence-to-sequence model, using a single, identical start symbol for both the source (encoder) and target (decoder) inputs would make it impossible for the model to distinguish between the two types of sequences and thus prevent it from learning the task.
Formulating NLP Tasks as Sequence-to-Sequence Mappings using Start Symbols