Short Answer

Calculating the Memory Summary Vector

A memory component is calculated as a weighted moving average of key vectors (k) and value vectors (v). The formula for the summary key vector is given by:

Summary Key = (Σ_{j=i-n_c+1}^{i} β_{j-i+n_c} k_j) / (Σ_{j=1}^{n_c} β_j)

Where i is the current position, n_c is the context window size, and β is a vector of weights.

Given the following:

  • Current position i = 5
  • Context window size n_c = 3
  • Weights β = [β_1, β_2, β_3] = [0.1, 0.3, 0.6]
  • Key vectors:
    • k_3 = [2, 4]
    • k_4 = [1, 3]
    • k_5 = [5, 0]

Calculate the summary key vector. Provide your answer as a vector (e.g., [x, y]).

0

1

Updated 2025-10-08

Contributors are:

Who are from:

Tags

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Application in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science