A language model is being trained on a task where it must determine if Sentence B is the actual sentence that follows Sentence A in a document. Which of the following training pairs is most likely to encourage the model to learn a simple, superficial shortcut for this task, rather than developing a deeper understanding of semantic coherence?

Simplicity of NSP Task as a Cause for Reliance on Superficial Cues

Diagnosing a Language Model's Flawed Coherence Judgment

Unintended Learning in Sentence Relationship Models