Construction of Top-K Candidate Sequences in Beam Search
After identifying the most probable next tokens at step , the set of top- candidate sequences is formed by appending each of these tokens to the parent sequence, . For each token in the set of top tokens (where ranges from 1 to ), a new candidate sequence is constructed as follows: This process expands a single parent hypothesis into new, longer hypotheses to be considered in the next step of the beam search.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Construction of Top-K Candidate Sequences in Beam Search
Mathematical Definition of Top-K Token Selection
A language model is generating text using a search algorithm. At a certain step, it has the partial sequence 'The cat sat on the' and calculates the following probabilities for the next word from its vocabulary:
Word Probability mat 0.45 rug 0.25 chair 0.15 floor 0.10 table 0.03 window 0.02 If the algorithm is configured to select the 3 most probable next words at this step, which set of words will be chosen to create new candidate sequences?
Debugging a Text Generation System
A text generation system is designed to explore multiple possible sentence continuations at each step. It does this by selecting a fixed number of the most probable next words from its entire vocabulary. Match each parameter setting or concept with its most likely consequence or definition.
At a certain step in a sequence generation process, the probabilities for the next token over a vocabulary V = {'A', 'B', 'C', 'D', 'E'} are as follows: Pr('A')=0.1, Pr('B')=0.4, Pr('C')=0.05, Pr('D')=0.3, Pr('E')=0.15. If the selection process is defined by the function
argTopKwith K=3, which set of tokens will be selected?Analyzing a Formalism for Token Selection
Construction of Top-K Candidate Sequences in Beam Search
Formula for Constructing Top-K Candidate Sequences
Evaluating a Token Selection Implementation
Learn After
Formula for the Candidate Set in Beam Search
In a text generation process, a single partial sequence is being expanded. The current sequence is 'The sun is shining', and the three most probable next words have been identified as 'brightly', 'today', and 'and'. Based on this information, what will be the new set of candidate sequences to consider for the next step?
Error Analysis in Sequence Expansion
In a text generation process, several partial sequences (parent hypotheses) are being considered. For each parent hypothesis, the three most probable next tokens have been identified. Match each parent hypothesis to its correctly constructed set of new, longer candidate sequences.