Learn Before
Behavior-Based Stopping Criteria
An approach to text generation termination involves designing stopping criteria based on the dynamic behavior of the LLM during inference. This category includes rules that monitor model outputs or predictive confidence, such as detecting repetition or a drop in token probability, to decide when to halt the process.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
End-of-Sequence (EOS) Token as a Stopping Criterion
Sequence Count as a Stopping Criterion in Beam Search
Maximum Output Length as a Stopping Criterion
Cost-Based Stopping Criteria
Behavior-Based Stopping Criteria
Debugging an Uncontrolled Text Generation System
A developer is testing a new text-generation system. They find that when prompted, the system produces a relevant initial response but then continues to generate a long, rambling stream of unrelated text until it is manually interrupted. What is the most fundamental problem with the system's configuration that leads to this behavior?
Consequences of Unbounded Text Generation
Learn After
Probability Threshold as a Stopping Criterion
Repetition Detection as a Stopping Criterion
An engineer is developing a chatbot designed for creative storytelling. To ensure the stories don't become nonsensical or repetitive, the engineer needs to implement a rule to stop the text generation process. Which of the following rules is based on monitoring the dynamic behavior of the model's output during generation?
Debugging a Text Generation Loop
Evaluating Stopping Criteria for Different Generation Tasks