Learn Before
Probability Threshold as a Stopping Criterion
A behavior-based stopping criterion can be implemented by halting the decoding process when the model's confidence in its next prediction drops. Specifically, generation is terminated if the probability of the next token falls below a predetermined threshold.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Probability Threshold as a Stopping Criterion
Repetition Detection as a Stopping Criterion
An engineer is developing a chatbot designed for creative storytelling. To ensure the stories don't become nonsensical or repetitive, the engineer needs to implement a rule to stop the text generation process. Which of the following rules is based on monitoring the dynamic behavior of the model's output during generation?
Debugging a Text Generation Loop
Evaluating Stopping Criteria for Different Generation Tasks
Learn After
A text generation system is configured to halt its output when the probability of the next predicted token drops below a specific value. Users report that the system frequently generates incomplete sentences, stopping abruptly in the middle of a thought. Which of the following is the most likely explanation for this behavior?
Applying a Probability Threshold Stopping Criterion
Evaluating the Probability Threshold Stopping Criterion