Example of Sentence Reordering in Denoising Autoencoding
Sentence reordering is a denoising autoencoding task that trains an encoder-decoder model to restore the logical flow of a multi-sentence input. For example, given a corrupted input where the sentences are swapped, such as [C] The ball rolls away swiftly . The kitten is chasing the ball ., the model must output the sentences in their original, coherent order: The kitten is chasing the ball . The ball rolls away swiftly ..
0
1
Tags
Foundations of Large Language Models
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Example of a Sentence Reordering Task
A language model developer observes that their model generates fluent, grammatically correct individual sentences. However, when generating multi-sentence paragraphs, the model's output often lacks logical coherence, with the order of events or ideas appearing random and jumbled. Which of the following training objectives would be most effective at directly addressing this specific weakness?
A language model is being trained using a method where it receives a modified version of a text and must learn to reconstruct the original. Consider the following example:
Original Text: 'The team gathered in the lab to review the experiment's results. Initial readings showed a significant breakthrough. They decided to run a second verification test immediately.'
Training Input: 'Initial readings showed a significant breakthrough. They decided to run a second verification test immediately. The team gathered in the lab to review the experiment's results.'
Based on this example, what is the primary skill this specific training method is designed to teach the model?
Evaluating a Training Strategy
Example of Sentence Reordering in Denoising Autoencoding