Learn Before
Maximum Output Length as a Stopping Criterion
To manage decoding costs and prevent excessive verbosity in practical applications, a common stopping criterion is to impose a maximum output length. Under this rule, the text generation process is halted as soon as the model generates a predefined number of tokens.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
End-of-Sequence (EOS) Token as a Stopping Criterion
Sequence Count as a Stopping Criterion in Beam Search
Maximum Output Length as a Stopping Criterion
Cost-Based Stopping Criteria
Behavior-Based Stopping Criteria
Debugging an Uncontrolled Text Generation System
A developer is testing a new text-generation system. They find that when prompted, the system produces a relevant initial response but then continues to generate a long, rambling stream of unrelated text until it is manually interrupted. What is the most fundamental problem with the system's configuration that leads to this behavior?
Consequences of Unbounded Text Generation
Learn After
Critique of a Fixed-Length Generation Rule
Controlling Chatbot Verbosity
A developer configures a language model for a customer support chatbot. To ensure responses are concise, they set the only stopping criterion to be a maximum output length of 30 tokens. Which of the following issues is most likely to occur as a direct result of this specific configuration?