Differentiating Masked Prediction Placeholders
A language model is given the context [C] The scientist observed and two different prediction tasks:
Task A: Complete the structure [M] the [M].
Task B: Complete the structure [X] the [Y].
Explain how the use of distinct placeholders ([X], [Y]) in Task B, compared to the repeated placeholder ([M]) in Task A, is likely to influence the relationship between the words the model predicts for the two blanks. Provide one plausible example prediction for each task to illustrate your point.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is tasked with completing the sentence structure
[X] with [Y]given the context[C] The chef prepared. The use of distinct placeholders,[X]and[Y], suggests that the model should predict two different but related components. Which of the following prediction pairs for([X], [Y])best fulfills this objective?Differentiating Masked Prediction Placeholders
Evaluating Model Predictions with Distinct Placeholders