Diagnosing Repetitive Text Generation
A language model is designed to summarize long articles. While the generated summaries are factually correct, they often contain repetitive phrases, making them sound unnatural. Explain how the standard search objective (finding the most probable sequence of words) might contribute to this issue and briefly describe, in principle, how modifying this objective could lead to more varied and natural-sounding summaries.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
MBR Decoding as an Alternative to MAP Decoding
Incorporating Penalty Terms for Controllable Decoding
Improving Generic Text Generation
A language model is tasked with generating creative story endings. Its current decoding process consistently produces endings that are grammatically perfect and logically sound, but are often predictable and repetitive (e.g., '...and they all lived happily ever after.'). Which of the following statements best analyzes why modifying the search objective could address this issue?
Diagnosing Repetitive Text Generation