Learn Before
Example of a Suboptimal Greedy Search Outcome
A greedy search sequence is not always optimal because early sub-optimal token choices can lead to higher subsequent probabilities. Suppose a greedy search yields the sequence "A", "B", "C", "
0
1
Tags
D2L
Dive into Deep Learning @ D2L
Related
Evaluating an Inference Strategy for a Creative Writing Assistant
A language model is generating a response, starting with the phrase 'The best way to learn is...'. The model must decide on the next three words. At each step, it uses a strategy where it always chooses the single word with the highest probability. Given the probabilities below, which sequence of three words will the model generate?
Step 1 (after '...is'):
by: 0.5through: 0.3with: 0.2
Step 2 (if 'by' is chosen):
doing: 0.6reading: 0.3listening: 0.1
Step 3 (if 'by doing' is chosen):
it: 0.4the: 0.5a: 0.1
Analyzing a Step-by-Step Text Generation Strategy
Example of a Suboptimal Greedy Search Outcome