Analyzing a Corrupted Input Sequence
A language model is being trained on the sentence: The chef prepared a delicious meal. During one training step, the input is modified to: The chef prepared a delicious apple. Explain what the model is expected to predict for the position of the word 'apple' and why this specific modification technique is used.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is being trained using a technique where some words in the input are altered to help the model learn. Consider the original input sequence:
[CLS] My dog chased the ball . [SEP] He brought it back . [SEP]. If the token 'ball' is selected to be replaced by a random word from the model's vocabulary, which of the following represents the most likely resulting sequence?Analyzing a Language Model's Training Method
Analyzing a Corrupted Input Sequence