A language model is tasked with predicting the missing words in a masked bilingual sentence pair. Arrange the following steps in the correct chronological order to describe how the model's encoder processes the input to generate the final representations used for this prediction.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Comprehension in Revised Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A model processes the following bilingual sequence where some words have been replaced by a special symbol:
[CLS] The cat sat on the [MASK] . [SEP] Le chat s'est assis sur le [MASK] . [SEP]. The model's encoder computes a final numerical representation (a hidden state) for every symbol in the sequence. Considering the final hidden state calculated for the first[MASK]symbol (in the English part), which statement best analyzes the information it contains?A language model is tasked with predicting the missing words in a masked bilingual sentence pair. Arrange the following steps in the correct chronological order to describe how the model's encoder processes the input to generate the final representations used for this prediction.
Analyzing Contextualization in Transformer Encoders