Debugging Input Representation in a Sequence-to-Sequence Model
Based on the information provided, analyze the potential cause of the model's poor performance and propose a specific modification to the input data format that would likely resolve the issue. Explain your reasoning.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Debugging Input Representation in a Sequence-to-Sequence Model
Example of a T5 Machine Translation Training Sample with Special Tokens
In designing a sequence-to-sequence model, an engineer decides to use one specific start symbol for all source sequences fed to the encoder and a different, unique start symbol for all target sequences fed to the decoder. Which statement best analyzes the primary benefit of this design choice?
In a sequence-to-sequence model, using a single, identical start symbol for both the source (encoder) and target (decoder) inputs would make it impossible for the model to distinguish between the two types of sequences and thus prevent it from learning the task.
Formulating NLP Tasks as Sequence-to-Sequence Mappings using Start Symbols