Google

This example illustrates the target sequence for a masked prediction task where the prediction order is specified. For an input like `[C] The kitten [X] the [Y]`, the model's goal is to generate the missing words in a set order. The target `[X]1 is2 chasing3 [Y]4 ball5 .` indicates that the model should predict 'is', then 'chasing' for the `[X]` placeholder, and subsequently 'ball' for the `[Y]` placeholder. The numbers define the explicit prediction sequence for the masked tokens.

Example of an Ordered Target Sequence for Masked Prediction

A language model is given the input `[C] An apple a day [X] the [Y] away.` and must be trained to generate the complete, well-known proverb. Construct the ordered target sequence that would correctly guide the model's predictions to fill in the placeholders.

Constructing an Ordered Target Sequence

A language model is given the input `[C] The server [X] the [Y] .` and the corresponding ordered target sequence `[X]1 brought2 [Y]3 check4`. Arrange the following actions the model must take to correctly generate the full sentence.

A language model is given the input `[C] The quick brown [X] jumps over the [Y] dog.` and the corresponding target sequence `[X]1 fox2 [Y]3 lazy4 .5`. Which statement correctly analyzes the specific task the model is being trained to perform?

Learn Before

Related