Learn Before
Example of a Simple Target Sequence
This example presents a simplified target sequence for a language modeling task. The sequence kitten is chasing ball represents the desired output from a model. This format is notable for its lack of punctuation, capitalization, and function words like articles, providing a basic, unformatted ground truth for model training or evaluation.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Example of an Ordered Target Sequence for Masked Prediction
A language model is presented with the following input text where a portion is hidden: 'The sun shines brightly in the [X] sky.' Which of the following tasks best represents the model's primary objective if it is performing a masked prediction task?
Example of Masked Prediction: Kitten Chasing
Example of Masked Language Modeling: Kitten Playing
Example of a Simple Target Sequence
Example of Masked Prediction with a Known Verb
Example of Masked Prediction with Distinct Placeholders
Choosing a Model Training Strategy
Selecting a Training Objective for a Grammar-Focused Model
Learn After
A language model is being trained. It receives the input context 'The playful kitten...' and needs to learn to generate the completion '...is chasing a red ball.' To create a basic, unformatted target sequence for this training example, which of the following options best represents the desired output?
Creating a Simple Target Sequence
A language model is being trained to generate the sentence 'The kitten is chasing the ball.' The sequence
The kitten is chasing the ball.is presented as the ground truth for this training example. This sequence is an appropriate example of a simple, unformatted target sequence.