Applying Masked Language Modeling
A language model is being trained using the sentence: 'The quick brown fox jumps over the lazy dog.' For one training instance, the model is presented with the following modified input: 'The quick [MASK] fox jumps over the [MASK] dog.' Based on this example, what is the model's primary task, and what specific words is it attempting to predict?
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is being trained on the sentence: 'The chef carefully seasoned the delicious soup with a pinch of salt.' Consider two different training examples created from this sentence:
... seasoned the delicious soup with a [MASK] of salt.The [MASK] carefully seasoned the delicious [MASK] with a pinch of salt.
Which statement best analyzes the difference in the learning objective between these two examples?
Applying Masking Strategies for Language Models
Applying Masked Language Modeling