Learn Before
Example of Masked Language Modeling: Kitten Playing
This example illustrates a masked language modeling task. The model is given an input sentence with missing words, structured as [C] The kitten [M] playing the [M] ., where [C] denotes the context and [M] represents the masked tokens. The objective is for the model to predict the masked words to complete the sentence, for example, 'The kitten is playing with the toy'.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Example of an Ordered Target Sequence for Masked Prediction
A language model is presented with the following input text where a portion is hidden: 'The sun shines brightly in the [X] sky.' Which of the following tasks best represents the model's primary objective if it is performing a masked prediction task?
Example of Masked Prediction: Kitten Chasing
Example of Masked Language Modeling: Kitten Playing
Example of a Simple Target Sequence
Example of Masked Prediction with a Known Verb
Example of Masked Prediction with Distinct Placeholders
Choosing a Model Training Strategy
Selecting a Training Objective for a Grammar-Focused Model
Learn After
A language model is tasked with filling in the missing words, marked as
[M], in the following sentence: "The kitten[M]playing with the[M]." Based on common linguistic patterns and world knowledge, which pair of words is the most probable prediction for the two[M]slots, respectively?Analyzing Context for Word Prediction
Evaluating Language Model Outputs for a Masked Sentence