In a masked language modeling task with the input The student opened their [M] to study for the exam., the model's prediction for the masked word is solely determined by the words that come before the mask (The student opened their).
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Evaluating Masking Strategies for Model Training
A language model is being trained using a masked language modeling objective. If the original input sentence is 'The innovative chef prepares a delicious meal for the guests.', which of the following masked versions would be the most difficult for the model to reconstruct accurately?
In a masked language modeling task with the input
The student opened their [M] to study for the exam., the model's prediction for the masked word is solely determined by the words that come before the mask (The student opened their).