Google

This example illustrates a masked prediction task using distinct placeholders, `[X]` and `[Y]`, to mark different missing parts of a sentence. Given the context `[C] The kitten`, the model's objective is to predict the words that replace `[X]` and `[Y]` in the structure `[X] the [Y]`.

Example of Masked Prediction with Distinct Placeholders

A language model is tasked with completing the sentence structure `[X] with [Y]` given the context `[C] The chef prepared`. The use of distinct placeholders, `[X]` and `[Y]`, suggests that the model should predict two different but related components. Which of the following prediction pairs for `([X], [Y])` best fulfills this objective?

A language model is given the context `[C] The scientist observed` and two different prediction tasks:

Task A: Complete the structure `[M] the [M]`.
Task B: Complete the structure `[X] the [Y]`.

Explain how the use of distinct placeholders (`[X]`, `[Y]`) in Task B, compared to the repeated placeholder (`[M]`) in Task A, is likely to influence the relationship between the words the model predicts for the two blanks. Provide one plausible example prediction for each task to illustrate your point.

Differentiating Masked Prediction Placeholders

A language model is given the context `[C] The musician played` and the structure `[X] on the [Y]`. The model's task is to predict a pair of words for `([X], [Y])` that are different but semantically related, as suggested by the distinct placeholders. Analyze the three outputs below and determine which one best fulfills the task's objective. Justify your choice by explaining why it is superior to the other two options.

Learn Before

Related