Formulating NLP Tasks as Sequence-to-Sequence Mappings using Start Symbols
Natural Language Processing tasks can be formulated as sequence-to-sequence mappings by employing specific start symbols to differentiate the input (source) from the output (target). For instance, the token is conventionally used as the start symbol on the source side, while is used as the start symbol on the target side. This unified representation allows diverse problems to be expressed in the exact same format.
0
1
Tags
Foundations of Large Language Models
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Debugging Input Representation in a Sequence-to-Sequence Model
Example of a T5 Machine Translation Training Sample with Special Tokens
In designing a sequence-to-sequence model, an engineer decides to use one specific start symbol for all source sequences fed to the encoder and a different, unique start symbol for all target sequences fed to the decoder. Which statement best analyzes the primary benefit of this design choice?
In a sequence-to-sequence model, using a single, identical start symbol for both the source (encoder) and target (decoder) inputs would make it impossible for the model to distinguish between the two types of sequences and thus prevent it from learning the task.
Formulating NLP Tasks as Sequence-to-Sequence Mappings using Start Symbols