Learn Before
Evaluating a Training Strategy for a Summarization Model
An engineer is training a model for the task of document summarization. They propose using only token reordering as the input corruption method during the pre-training phase. Their reasoning is that this will force the model to understand the core topics of a document regardless of sentence structure. Evaluate this training strategy. In your evaluation, discuss at least one significant advantage and one significant disadvantage of this approach for the specific goal of creating a high-quality summarization model.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
An engineer is training a model whose task is to reconstruct an original sentence from a modified version of it. The engineer's primary goal is to force the model to learn the semantic meaning of the sentence, independent of the specific ordering of its words. Which of the following modification techniques, when applied to the input sentence, would be most effective for achieving this specific training objective?
Comparing Input Alteration Techniques
Evaluating a Training Strategy for a Summarization Model
Example of Token Reordering in Denoising Autoencoding