Role of the Adapter in BERT-based NMT
In a neural machine translation system that uses a pre-trained, frozen BERT model as the encoder and a randomly initialized transformer model as the decoder, an 'adapter' layer is often placed between these two components. Explain the primary technical reason for including this adapter layer and describe one potential negative consequence of omitting it.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
An engineer is developing a system to translate text from a source language to a target language. The system uses a large, pre-trained model as an 'encoder' to process the source sentence and create a rich, contextual numerical representation. A separate, newly trained 'decoder' component then uses this representation to generate the translated sentence. During testing, the engineer observes that the generated sentences in the target language are grammatically fluent and well-structured, but they frequently fail to accurately convey the specific meaning and context of the original source sentences. Which of the following is the most likely cause of this specific problem?
Evaluating Encoder Choices in Machine Translation
Role of the Adapter in BERT-based NMT