A language model developer observes that their model generates fluent, grammatically correct individual sentences. However, when generating multi-sentence paragraphs, the model's output often lacks logical coherence, with the order of events or ideas appearing random and jumbled. Which of the following training objectives would be most effective at directly addressing this specific weakness?
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Example of a Sentence Reordering Task
A language model developer observes that their model generates fluent, grammatically correct individual sentences. However, when generating multi-sentence paragraphs, the model's output often lacks logical coherence, with the order of events or ideas appearing random and jumbled. Which of the following training objectives would be most effective at directly addressing this specific weakness?
A language model is being trained using a method where it receives a modified version of a text and must learn to reconstruct the original. Consider the following example:
Original Text: 'The team gathered in the lab to review the experiment's results. Initial readings showed a significant breakthrough. They decided to run a second verification test immediately.'
Training Input: 'Initial readings showed a significant breakthrough. They decided to run a second verification test immediately. The team gathered in the lab to review the experiment's results.'
Based on this example, what is the primary skill this specific training method is designed to teach the model?
Evaluating a Training Strategy
Example of Sentence Reordering in Denoising Autoencoding