Learn Before
Example of an Ordered Target Sequence for Masked Prediction
This example illustrates the target sequence for a masked prediction task where the prediction order is specified. For an input like [C] The kitten [X] the [Y], the model's goal is to generate the missing words in a set order. The target [X]1 is2 chasing3 [Y]4 ball5 . indicates that the model should predict 'is', then 'chasing' for the [X] placeholder, and subsequently 'ball' for the [Y] placeholder. The numbers define the explicit prediction sequence for the masked tokens.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Example of an Ordered Target Sequence for Masked Prediction
A language model is presented with the following input text where a portion is hidden: 'The sun shines brightly in the [X] sky.' Which of the following tasks best represents the model's primary objective if it is performing a masked prediction task?
Example of Masked Prediction: Kitten Chasing
Example of Masked Language Modeling: Kitten Playing
Example of a Simple Target Sequence
Example of Masked Prediction with a Known Verb
Example of Masked Prediction with Distinct Placeholders
Choosing a Model Training Strategy
Selecting a Training Objective for a Grammar-Focused Model
Learn After
A language model is given the input
[C] The quick brown [X] jumps over the [Y] dog.and the corresponding target sequence[X]1 fox2 [Y]3 lazy4 .5. Which statement correctly analyzes the specific task the model is being trained to perform?Constructing an Ordered Target Sequence
A language model is given the input
[C] The server [X] the [Y] .and the corresponding ordered target sequence[X]1 brought2 [Y]3 check4. Arrange the following actions the model must take to correctly generate the full sentence.