Learn Before
Example of Masked Prediction: Kitten Chasing
This example demonstrates a masked prediction task where a model is provided with the input [C] The kitten [M] chasing the [M] .. The [C] token represents the context, while [M] signifies masked words. The model's objective is to predict the missing words, such as 'is' and 'ball', to form a coherent sentence.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Example of an Ordered Target Sequence for Masked Prediction
A language model is presented with the following input text where a portion is hidden: 'The sun shines brightly in the [X] sky.' Which of the following tasks best represents the model's primary objective if it is performing a masked prediction task?
Example of Masked Prediction: Kitten Chasing
Example of Masked Language Modeling: Kitten Playing
Example of a Simple Target Sequence
Example of Masked Prediction with a Known Verb
Example of Masked Prediction with Distinct Placeholders
Choosing a Model Training Strategy
Selecting a Training Objective for a Grammar-Focused Model
Learn After
A language model is tasked with filling in the missing words, marked as
[M], in the following sentence: 'The chef carefully [M] the fresh herbs before adding them to the [M].' Based on the surrounding words, which pair of words represents the most probable and contextually coherent completion of the sentence?Analyzing a Masked Prediction Task
A language model is presented with the following input containing two masked words, denoted by
[M]:The dog [M] the mailman down the [M] .The model's task is to predict the most likely words for the[M]placeholders based on the surrounding context. Which of the following predicted pairs best demonstrates the model's ability to understand the semantic relationship between the words in the sentence to form a coherent and probable event?