Multiple Choice

A model computes a memory component, Mem, using the following formula for a weighted moving average of the last n_c key (k) and value (v) vectors at a given position i:

Mem = ( (Σ_{j=i-n_c+1}^{i} β_{j-i+n_c} k_j) / (Σ_{j=1}^{n_c} β_j), (Σ_{j=i-n_c+1}^{i} β_{j-i+n_c} v_j) / (Σ_{j=1}^{n_c} β_j) )

Given a current position i=10, a context window size n_c=4, and weights β = [β_1, β_2, β_3, β_4], which of the following expressions correctly represents the calculation for the summary key vector?

0

1

Updated 2025-09-26

Contributors are:

Who are from:

Tags

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Application in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science