When a Transformer model is fine-tuned to compress a long context by sequentially processing text segments, it updates a memory state at each step. What is the most critical function of incorporating the memory state from the previous step when encoding the current text segment?
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Evaluating a Proposed Modification to a Sequential Processing Model
A Transformer model is adapted to compress a long text by processing it sequentially in segments. Arrange the following steps to accurately describe how this model iteratively builds a complete representation of the text.
When a Transformer model is fine-tuned to compress a long context by sequentially processing text segments, it updates a memory state at each step. What is the most critical function of incorporating the memory state from the previous step when encoding the current text segment?