Formula for Sequence Extension with a Sampled Token
In autoregressive text generation, a new sequence is formed at each step by appending a newly generated token to the existing sequence. If a token is sampled at step , the new sequence, denoted , is constructed by concatenating the preceding sequence with this sampled token. This process is formally represented by the equation:
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
An autoregressive language model has generated the sequence of tokens: 'The quick brown fox'. It is now about to generate the next token. Which expression accurately describes how the model will select this next token?
An autoregressive model selects the next token by sampling from a conditional probability distribution, represented by the formula: Match each component of this formula to its correct description.
Explaining Model Output Variability
Formula for Sequence Extension with a Sampled Token
Learn After
In an autoregressive text generation process, the sequence generated up to a certain point is
The dog chased the. At the current step, the model generates and selects the tokenball. What is the new, extended sequence that will be used as the basis for generating the subsequent token?An autoregressive model is generating a sequence. It begins with the single token
y_1= 'The'. In the next step, it samples the tokenȳ_2= 'cat'. Following that, it samples the tokenȳ_3= 'sat'. What is the resulting sequence that is formed after these two sampling steps?Formal Representation of Sequence Extension
Formula for the Candidate Set in Sampling-Based Decoding