Learn Before
Inferring Search Strategy from LLM Output
Analyze the two outputs in the case study below. Which output likely resulted from a more exhaustive search of possible sequences, and why? Your explanation should focus on the characteristics of the generated text as evidence for the underlying search process.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Decoder
A language model is generating a sentence and considers two different methods for choosing the sequence of words:
- Method A: At each step, the model selects the single most probable word and adds it to the sequence before moving to the next step.
- Method B: At each step, the model keeps track of the three most probable partial sentences generated so far, extends each of them with their most likely next words, and then keeps the three best resulting sentences to continue the process.
Which statement best analyzes the fundamental trade-off between these two methods in the context of finding the best possible output sequence?
Inferring Search Strategy from LLM Output
Explaining the Search Problem in Text Generation