Learn Before
Pruning and K-Best Output in Beam Search
Following the ranking stage in beam search, a selection is made based on the beam width, denoted as K. The top K candidates with the highest probabilities are kept as the output, while all other lower-probability candidates are discarded or 'pruned'. For instance, with a beam width of K=3, the top three candidates ('cute', 'on', 'sick') would form the output, and the remaining candidates ('are', '.') would be pruned.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Pruning and K-Best Output in Beam Search
A language model is generating the next word in a sequence. After considering several possibilities, it has calculated the following probabilities for five potential next tokens. Arrange these tokens in the correct order, from the most probable to the least probable, as would be done during the ranking stage of a search process.
Debugging a Generative Model's Selection Process
A language model is generating a sequence and has produced the following potential next tokens with their associated probabilities:
{'on': 0.32, 'cute': 0.34, 'sick': 0.21, '.': 0.01, 'are': 0.12}. Which of the following lists represents the correctly sorted output of the ranking stage for these candidates?
Learn After
In a text generation process, a set of potential next words and their calculated probabilities are ranked as follows:
{'the': 0.45, 'a': 0.25, 'his': 0.15, 'her': 0.10, 'its': 0.05}. If the process uses a fixed width of K=3 to select the most likely candidates, which words are kept and which are discarded (pruned)?Impact of Search Width on Text Generation
Applying Selection and Pruning in Text Generation