Reconstructing an Original Input Sequence
A language model is presented with the following modified input: [CLS] The sky is [MASK] . [SEP] The grass is [MASK] . [SEP]. The model's task is to predict the original words, which were 'blue' and 'green', respectively. What was the complete, original input sequence before the words were replaced?
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is being prepared to predict missing words in a two-sentence input. The original input is:
[CLS] The sun is shining brightly . [SEP] Birds are singing loudly . [SEP]. If the words 'shining' and 'singing' are selected for masking, which of the following represents the correctly modified input sequence?Reconstructing an Original Input Sequence
A researcher is preparing the input sequence
[CLS] The cat sat on the mat . [SEP]for a masked language modeling task. To train the model to predict the word 'sat', the researcher replaces 'sat' with the special token ____.