Learn Before
Formula for FIFO Update of Compressive Memory
The update process for the compressive memory, , operates as a First-In, First-Out (FIFO) queue. The function takes the newly compressed key-value pairs, , appends them to the queue, and drops the oldest entries from the previous compressive memory state, . This produces the updated memory state according to the formula:
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Related
Formula for FIFO Update of Compressive Memory
A compressive memory component is maintained as a fixed-size queue that can hold a maximum of 4 items. The current state of the memory is
[Item_1, Item_2, Item_3, Item_4], whereItem_1is the oldest item. If a new compressed item,Item_5, is added to this memory, what will be its resulting state?Compressive Memory Update Anomaly
A compressive memory component is implemented as a queue with a maximum capacity of 3 items. Initially, it contains
[C1, C2, C3], whereC1is the oldest item. Subsequently, two new compressed items,C4and thenC5, are added to the memory one after the other. Arrange the items that are removed from the memory in the correct chronological order of their removal.
Learn After
A model's memory component has a fixed capacity of 4 slots and is updated using a First-In, First-Out (FIFO) queue mechanism. The current state of the memory, ordered from oldest to newest, is
[M1, M2, M3, M4]. If a new set of 2 items,[N1, N2], is to be added, what will be the final state of the memory after the update?Deconstructing a Memory Update
The formula
CMem = FIFO(C_kv^k, CMem_pre)describes the update process for a fixed-size memory queue. Arrange the following events into the correct chronological order that this formula represents.