Learn Before
Analyzing a Training Method for Language Models
Based on the researcher's goal and proposed training method described in the case study, analyze the primary advantage of using an input structure with a known, unmasked verb. How does this specific structure provide a more targeted test of grammatical understanding compared to a structure where all words following the subject are masked?
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is given the task of predicting the words for the masked tokens
[M]in the following input:The car [M] was [M] by the mechanic.Which of the following pairs of predictions for the two[M]tokens demonstrates the strongest understanding of how the known words ('was', 'by') constrain the grammatical structure of the sentence?Analyzing Grammatical Constraints in Masked Prediction
Analyzing a Training Method for Language Models