Diagnosing a Failing Sequence Generation Algorithm
Based on the provided scenario, what specific modification should be made to the engineer's formula to make the generation process computationally feasible? Explain why this modification solves the problem of the system running out of memory.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
In a sequence generation process, the set of candidate sequences at step
i, denotedY_i, is generated from the previous setY_{i-1}and the entire vocabularyV. Consider the difference between two methods for generatingY_i:Method A:
Y_i = Y_{i-1} × VMethod B:Y_i = Prune(Y_{i-1} × V)What is the most significant practical difference in the outcome of using Method B instead of Method A, particularly for generating longer sequences?
Diagnosing a Failing Sequence Generation Algorithm
Applying Pruning in Sequence Generation