Learn Before
End-of-Sequence (EOS) Token as a Stopping Criterion
A common and straightforward stopping strategy in LLM inference is to terminate the generation process upon the production of a special end-of-sequence (EOS) token, such as ⟨EOS⟩ or ⟨/s⟩. Models are specifically trained to output this token to indicate that the generated text is complete.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
End-of-Sequence (EOS) Token as a Stopping Criterion
Sequence Count as a Stopping Criterion in Beam Search
Maximum Output Length as a Stopping Criterion
Cost-Based Stopping Criteria
Behavior-Based Stopping Criteria
Debugging an Uncontrolled Text Generation System
A developer is testing a new text-generation system. They find that when prompted, the system produces a relevant initial response but then continues to generate a long, rambling stream of unrelated text until it is manually interrupted. What is the most fundamental problem with the system's configuration that leads to this behavior?
Consequences of Unbounded Text Generation
Learn After
A developer is using a text-generation model to complete the sentence: 'The capital of France is'. The model produces the single word 'Paris' and then immediately stops. The developer had configured the generation process to allow for a maximum of 100 new words and is surprised by the short output. Based on how these models are trained to signal completeness, what is the most likely reason the generation process terminated after just one word?
Consequences of Training Data Omissions
Debugging Premature Text Generation Termination