Learn Before
Debugging a Repetitive Generation Loop
A developer is implementing a text generation loop for an auto-regressive language model. They notice that when they provide the input 'The ocean is vast and', the model produces the output 'deep deep deep deep...'. Based on your understanding of the iterative generation process, analyze this behavior and identify the most probable error in the developer's implementation of the generation loop.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
An auto-regressive language model is provided with the input sequence 'The sun is shining and the'. The model's next step is to generate the token ' birds'. To predict the token that will come after ' birds', what complete sequence will the model use as its new input?
An auto-regressive language model generates text one token at a time. Arrange the following actions into the correct chronological sequence that describes the process of generating one token and preparing for the next.
Debugging a Repetitive Generation Loop