Applying Sentinel Tokens to Masked Spans
A sentence has two sections identified for masking: 'The delivery driver dropped off the [large, heavy package] at the [front door].' Transform this sentence by replacing each of the two bracketed spans with its own unique sentinel token (e.g., [X], [Y]).
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Example of Target Output Format for Span-Based Denoising
Consider the following sentence where two spans of text have been selected for masking: 'The team celebrated after their [innovative new model] achieved [state-of-the-art results].' How would this sentence be represented if unique sentinel tokens are used to replace each complete masked span?
Match each original sentence, with text to be masked enclosed in brackets, to its correct representation using unique sentinel tokens.
Applying Sentinel Tokens to Masked Spans