Learn Before
Example of Masked Prediction with Distinct Placeholders
This example illustrates a masked prediction task using distinct placeholders, [X] and [Y], to mark different missing parts of a sentence. Given the context [C] The kitten, the model's objective is to predict the words that replace [X] and [Y] in the structure [X] the [Y].
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Example of an Ordered Target Sequence for Masked Prediction
A language model is presented with the following input text where a portion is hidden: 'The sun shines brightly in the [X] sky.' Which of the following tasks best represents the model's primary objective if it is performing a masked prediction task?
Example of Masked Prediction: Kitten Chasing
Example of Masked Language Modeling: Kitten Playing
Example of a Simple Target Sequence
Example of Masked Prediction with a Known Verb
Example of Masked Prediction with Distinct Placeholders
Choosing a Model Training Strategy
Selecting a Training Objective for a Grammar-Focused Model
Learn After
A language model is tasked with completing the sentence structure
[X] with [Y]given the context[C] The chef prepared. The use of distinct placeholders,[X]and[Y], suggests that the model should predict two different but related components. Which of the following prediction pairs for([X], [Y])best fulfills this objective?Differentiating Masked Prediction Placeholders
Evaluating Model Predictions with Distinct Placeholders