Based on your understanding of the token processing pipeline, identify the fundamental error in the data scientist's sorting step and explain why this error would negatively impact the quality of the generated text.

Google

The second stage in the Top-p sampling process involves ranking the candidate tokens generated during the expansion phase. These tokens are sorted in descending order according to their probabilities. For example, the token 'cute' with a probability of 0.34 is ranked highest, followed by 'on' with a probability of 0.32, 'sick' with 0.21, and so on.

Ranking Stage in Top-p Sampling

A language model has generated the following potential next tokens and their associated probabilities. Arrange these tokens in the correct order as they would appear after being sorted in descending order based on their probability.

A language model has produced a set of potential next tokens with their corresponding probabilities: 'cat' (0.22), 'dog' (0.35), 'bird' (0.15), 'fish' (0.18), and 'hamster' (0.10). Which option below correctly shows these tokens sorted in descending order of their probability?

Learn Before

Related