Learn Before
Analyzing Grammatical Constraints in Masked Prediction
A language model is given the input The scientist [M] is [M] a new theory. and generates two potential pairs of predictions for the masked tokens [M]:
Prediction A: ('developing', 'testing')
Prediction B: ('developed', 'tested')
Identify which prediction pair (A or B) is grammatically superior and explain why the presence of the word 'is' in the input makes that pair a better choice.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is given the task of predicting the words for the masked tokens
[M]in the following input:The car [M] was [M] by the mechanic.Which of the following pairs of predictions for the two[M]tokens demonstrates the strongest understanding of how the known words ('was', 'by') constrain the grammatical structure of the sentence?Analyzing Grammatical Constraints in Masked Prediction
Analyzing a Training Method for Language Models