Example of Span Masking

Prediction Challenge in Span Masking

An engineer is analyzing a text corruption technique and observes the following input-output pair:

Original Text: 'The very happy dog played in the big yard.' Processed Text: 'The [MASK] dog played [MASK] yard.'

Based on this single example, what can be definitively concluded about the rules of this corruption technique?

Reconstructing Masked Spans

Consider a text corruption technique where non-overlapping segments of text are randomly selected and each chosen segment is replaced by a single [MASK] token. According to this technique, if the three-word segment 'the big dog' is selected from a sentence, it would be replaced by '[MASK] [MASK] [MASK]'.

A text corruption technique involves selecting one or more non-overlapping segments of text (spans) and replacing each entire segment with a single [MASK] token. This technique also allows for selecting zero-length spans, which results in inserting a [MASK] token. Given the original sentence: 'The quick brown fox jumps over the lazy dog.', which of the following outputs represents an invalid application of this technique?