Decoder

A language model is generating a sentence and considers two different methods for choosing the sequence of words:

• Method A: At each step, the model selects the single most probable word and adds it to the sequence before moving to the next step.
• Method B: At each step, the model keeps track of the three most probable partial sentences generated so far, extends each of them with their most likely next words, and then keeps the three best resulting sentences to continue the process.

Which statement best analyzes the fundamental trade-off between these two methods in the context of finding the best possible output sequence?

Inferring Search Strategy from LLM Output

Explaining the Search Problem in Text Generation