Google

In the Compressive Transformer, the number of key-value pairs after compression, denoted as $n_{cs}$, is calculated by dividing the number of key-value pairs popped from the local memory, $n_s$, by a compression ratio, $c$. This relationship is expressed by the formula: $$ n_{cs} = \frac{n_s}{c} $$

Formula for Number of Compressed Key-Value Pairs

Given the following memory management scenario, calculate the number of compressed data items that will be added to the long-term archive. Explain your reasoning.

Memory Compression Scenario

A transformer model with a compressive memory system needs to update its long-term storage. It takes 120 key-value pairs from its short-term memory and applies a compression function with a ratio of 4. How many new key-value pairs will be added to the long-term compressive memory after this operation?

A memory compression system in a transformer model processes 200 key-value pairs from its short-term memory. After this process, 50 compressed key-value pairs are stored in its long-term memory. What was the compression ratio applied? Explain your calculation.

Learn Before

Related