Google

In top-K decoding strategies like beam search, the set of all candidate sequences at step `i` is denoted by $Y_i$. This set is composed of the $K$ new sequences that were formed by appending each of the top-K next tokens to the shared preceding sequence. The formal definition of this set is: $$Y_i = \{\mathbf{y}^{\text{top1}}, \dots, \mathbf{y}^{\text{topK}}\}$$

Formula for the Candidate Set in Top-K Decoding

A text generation model has produced the sequence of tokens `('The', 'quick', 'brown')`. At the current step, the model calculates the probabilities for the next token and identifies the top 3 most likely tokens as `('fox', 'dog', 'wolf')`, in that order of probability. Given this information, what is the resulting set of candidate sequences?

In a text generation process at step `i`, the candidate set $Y_i$ is defined as the single most probable sequence formed by appending the most likely next token to the shared preceding sequence.

A language model is generating a sequence. At step `i=4`, the shared preceding sequence is `('The', 'cat', 'sat')`. The model has identified the top-K next tokens (where K=3) as `y_4^top1 = 'on'`, `y_4^top2 = 'by'`, and `y_4^top3 = 'near'`. Using standard set notation, write out the complete candidate set, denoted as `Y_4`.

Learn Before

Related