BERT-based Encoder-Decoder for Neural Machine Translation
A neural machine translation (NMT) system serves as a specific application of the BERT-based encoder-decoder architecture. In this setup, a pre-trained BERT model functions as the encoder to process the source language text. The resulting contextual representation is then passed through an optional adapter to a decoder, which in turn generates the translated text in the target language.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.1 Pre-training - Foundations of Large Language Models
Related
Role of the Adapter in BERT-based Encoder-Decoder Models
Notation in a BERT-based Encoder-Decoder Architecture
BERT-based Encoder-Decoder for Neural Machine Translation
A developer is explaining the process of generating a target text sequence using an architecture composed of a pre-trained encoder and a separate decoder. Analyze the following statements from their explanation. Which statement incorrectly describes the relationship between the encoder's output and the decoder's input during the generation process?
A sequence-to-sequence model uses a pre-trained text model as its encoder and a separate model as its decoder. Arrange the following steps to accurately represent the data flow from the initial source text to the final generated target text.
Diagnosing an Encoder-Decoder Model Failure
Learn After
An engineer is developing a system to translate text from a source language to a target language. The system uses a large, pre-trained model as an 'encoder' to process the source sentence and create a rich, contextual numerical representation. A separate, newly trained 'decoder' component then uses this representation to generate the translated sentence. During testing, the engineer observes that the generated sentences in the target language are grammatically fluent and well-structured, but they frequently fail to accurately convey the specific meaning and context of the original source sentences. Which of the following is the most likely cause of this specific problem?
Evaluating Encoder Choices in Machine Translation
Role of the Adapter in BERT-based NMT