Evaluating a Proposed Modification to a Sequential Processing Model
A team is using a specific method to create a compact representation of a very long document. The method works by breaking the document into sequential segments. A model processes these segments one by one. For each segment, it generates an updated 'memory state' by considering both the previous memory state and the current text segment. The final memory state, after the last segment is processed, represents the entire document.
A new team member suggests an optimization: to make the process faster, the model should generate each new memory state by looking only at the current text segment, ignoring the previous memory state. Evaluate this proposed optimization. Is it a valid approach for creating a comprehensive representation of the entire document? Justify your conclusion.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Evaluating a Proposed Modification to a Sequential Processing Model
A Transformer model is adapted to compress a long text by processing it sequentially in segments. Arrange the following steps to accurately describe how this model iteratively builds a complete representation of the text.
When a Transformer model is fine-tuned to compress a long context by sequentially processing text segments, it updates a memory state at each step. What is the most critical function of incorporating the memory state from the previous step when encoding the current text segment?