Learn Before
Auto-Encoding (AE) Models
This type of LM destroys the input text (e.g. masking words in sentence and trying to reconstruct the original text). It aims to build bidirectional encoding representations of the entire sentences, so infrastructures often correspond to the encoder part of transformer, where all input can be accessed at each location. They have been fine-tuned successfully for downstream tasks. Examples include BERT, ROBERTA, ERNIE, and applications are suitable for NLU tasks like sentence classification and sequence labeling.
0
1
Tags
Deep Learning (in Machine learning)
Data Science
Related
Auto-Encoding (AE) Models
Auto-Regressive (AR) Models
Seq2seq Models for Text Generation
An engineering team is tasked with creating a system to analyze customer reviews and automatically classify them as 'positive', 'negative', or 'neutral'. The most critical requirement is for the model to have a deep, holistic understanding of the entire review's context to make an accurate classification. Which of the following architectural approaches for a pre-trained model would be best suited for this task?
You are an NLP engineer selecting a pre-trained model architecture for three different projects. Match each project description to the most suitable underlying model training objective.
Model Architecture Selection Flaw