Learn Before
Consequences of Over-Tuning a Repetition Penalty
A developer is using a text generation model to summarize articles. They notice the summaries often contain repetitive phrases like 'The article discusses... The article also discusses...'. To fix this, they apply a strong penalty to tokens that have already appeared in the generated text. Describe two potential negative consequences of setting this penalty value excessively high.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Representation-based Repetition Penalty
A language model is tasked with writing a short story and produces the following output: 'The knight rode his horse through the dark forest. The forest was very dark. The knight was a brave knight, and the dark forest did not scare him.' Which of the following adjustments to the generation process would be most effective at discouraging this kind of repetitive phrasing?
Tuning Text Generation for Poetry
Consequences of Over-Tuning a Repetition Penalty