Learn Before
General Formula for Recurrent Memory Update
In the general formulation of recurrent memory update, and represent the memory outputs at the current and previous steps, respectively. denotes a set of key-value pairs representing the model's recent states. The general relationship is expressed as: . By selecting different configurations for the function and the set , various memory models can be developed.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
General Formula for Recurrent Memory Update
Cumulative Average of Keys and Values for Memory Component
Recurrent Network as a Cache Mechanism
A system is designed to process an extremely long, continuous sequence of information. To manage this, it uses a memory cache that is updated at each step: a new key-value pair is combined with the entire compressed memory from the previous step to form a new, equally compressed memory state. What is the primary trade-off inherent in this design?
A system maintains a fixed-size memory cache by processing a sequence of key-value pairs one at a time. Arrange the following events in the correct chronological order for a single update step.
Memory Cache State Calculation
Learn After
A system is designed to maintain a compressed summary of information over a sequence of inputs. At each step, it updates its state using the general relationship:
New_Summary = Update_Function(Current_Input, Previous_Summary). What is the most significant implication of structuring the update process in this recurrent manner?Applying Recurrent Memory Update to Network Monitoring
Calculating a New Memory State