Step-by-Step Memory State Calculation
A system updates its memory state by calculating the cumulative average of all key-value pairs it has processed. It uses the following recursive formula, where i is the index of the current update step (e.g., for the second item, i=1):
Mem_i = ((k_i, v_i) + i * Mem_{i-1}) / (i+1)
Given the sequence of inputs below, calculate the final memory state Mem_2.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Memory Efficiency of Recursive Cumulative Average
A system processes a very long sequence of data pairs, one at a time. At each step, it must update a 'memory state' to be the precise mathematical average of all data pairs seen up to that point. Consider two methods for updating the memory state at the 10,000th step:
Method A: Re-access the entire history of 10,000 data pairs, sum them up, and divide by 10,000.
Method B: Use only the memory state from the 9,999th step (which was the average of the first 9,999 pairs) and the new 10,000th data pair to calculate the new average.
Which statement best analyzes the primary advantage of Method B over Method A in this context?
Calculating a Recursive Memory State
Step-by-Step Memory State Calculation
Inference Efficiency of Cumulative Average Memory