Google

In top-$$K$$ decoding strategies like beam search, the set of all candidate sequences at step $$i$$ is denoted by $$Y_i$$. This set is composed of the $$K$$ new sequences that were formed by appending each of the top-$$K$$ next tokens to the shared preceding sequence. The formal definition of this set is:

$$Y_i = \{\mathbf{y}^{\text{top}1}, \dots, \mathbf{y}^{\text{top}K}\}$$

Formula for the Candidate Set in Top-K Decoding

In a text generation process at step `i`, the candidate set $Y_i$ is defined as the single most probable sequence formed by appending the most likely next token to the shared preceding sequence.

A language model is generating a sequence. At step `i=4`, the shared preceding sequence is `('The', 'cat', 'sat')`. The model has identified the top-K next tokens (where K=3) as `y_4^top1 = 'on'`, `y_4^top2 = 'by'`, and `y_4^top3 = 'near'`. Using standard set notation, write out the complete candidate set, denoted as `Y_4`.

Defining the Candidate Set in Top-K Decoding

A text generation model has produced the sequence of tokens `('The', 'quick', 'brown')`. At the current step, the model calculates the probabilities for the next token and identifies the top 3 most likely tokens as `('fox', 'dog', 'wolf')`, in that order of probability. Given this information, what is the resulting set of candidate sequences?

Learn Before

Related