Analyzing Generator Output in Text Corruption
A text-corrupting model is given the masked sentence: 'The chef prepared the [MASK] dish with [MASK] ingredients.' The original, unmasked sentence was: 'The chef prepared the signature dish with fresh ingredients.' The model outputs the following new sentence: 'The chef prepared the delicious dish with local ingredients.' Based on the function of this type of model, explain why the output tokens ('delicious', 'local') can be different from the original tokens ('signature', 'fresh').
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A small masked language model, acting as a generator, is given an input sequence where some tokens have been replaced with a [MASK] token: 'The quick [MASK] fox jumps over the [MASK] dog.' The original, unmasked sentence was: 'The quick brown fox jumps over the lazy dog.' Based on the typical behavior of such a generator, which of the following is the most plausible output sequence it would produce to be passed on for further processing?
Analyzing Generator Output in Text Corruption
The generator's role in replaced token detection involves several steps to create a plausible but altered version of an input sentence. Arrange the following events into the correct chronological sequence.