Debugging Draft Token Generation
Analyze the engineer's implementation described in the case study. Identify the fundamental error in their logic regarding how the sequence of draft tokens is produced, and briefly describe the correct procedure.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A small, fast language model is used to generate a sequence of 3 candidate words. Starting with the initial context 'The cat sat on the', arrange the following actions in the correct chronological order to produce the three-word draft sequence.
A small, fast language model is used to generate a sequence of 3 candidate tokens. It generates one token at a time, and each new token prediction is based on the original context plus any tokens it has already generated in the current sequence. Given the model's predictions below, what is the final 3-token sequence it will generate starting from the context 'The ocean is'?
- When given 'The ocean is', the model's top prediction is 'blue'.
- When given 'The ocean is blue', the model's top prediction is 'and'.
- When given 'The ocean is blue and', the model's top prediction is 'deep'.
- When given 'The ocean is deep', the model's top prediction is 'water'.
Debugging Draft Token Generation