A language model has generated a set of candidate tokens to follow a sequence, each with an assigned probability. Arrange these candidates in the correct descending order based on their probability, as would occur during the ranking stage of a selection process.

A language model generates the following five candidate tokens and their associated probabilities to complete a sentence: 'sky' (Pr=0.45), 'moon' (Pr=0.15), 'clouds' (Pr=0.35), 'stars' (Pr=0.04), 'blue' (Pr=0.01). What is the primary purpose of sorting these candidates by their probability in a process designed to select only the single most likely token?

Interpreting Model Certainty from Ranked Probabilities