Short Answer

Calculating a Memory State

A memory mechanism updates its state after each token by maintaining a tuple of the cumulative average of all key vectors and all value vectors seen so far. Given the following sequence of 2-dimensional key (k) and value (v) vectors, calculate the final memory state Mem after processing the token at index i=2.

  • i=0: k_0 = [2, 4], v_0 = [1, 0]
  • i=1: k_1 = [6, 2], v_1 = [3, 2]
  • i=2: k_2 = [1, 0], v_2 = [5, 1]

Express your answer as a tuple of two vectors, e.g., ([x1, y1], [x2, y2]).

0

1

Updated 2025-10-08

Contributors are:

Who are from:

Tags

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Application in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science