Learn Before
Addressing Repetitive Model Outputs
A developer is using a language model with a search-based decoding strategy to generate multiple creative story ideas from a single prompt. They observe that the generated outputs, while grammatically correct, are often minor variations of the same core concept, lacking true creative divergence. For example, for the prompt 'An astronaut makes a startling discovery on a new planet,' the model consistently produces outputs centered around finding simple alien life forms. How could the developer modify the decoding process to address this issue and produce a more varied set of story ideas? Explain the reasoning behind your proposed solution.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model using a search algorithm is prompted to generate three distinct completions for the sentence: 'The most rewarding part of learning a new skill is...'. Which of the following sets of completions most likely had a diversity penalty applied during the generation process?
Addressing Repetitive Model Outputs
Evaluating the Use of a Diversity Penalty in a Creative Writing Assistant