Example

RNN-like Configuration for Neural Memory

A general neural network memory formulation, Mem=Update(Skv,Mempre)\mathrm{Mem} = \mathrm{Update}(S_{\mathrm{kv}}, \mathrm{Mem}_{\mathrm{pre}}), can be configured to act as an RNN-like model. This occurs when the set of recent model states, SkvS_{\mathrm{kv}}, is restricted to contain only the most recent key-value pair, (ki,vi)(\mathbf{k}_i, \mathbf{v}_i), and the Update()\mathrm{Update}(\cdot) mechanism is defined as a recurrent cell, denoted as ff. This specific configuration can be expressed with the formula:

Mem=f((ki,vi),Mempre)\mathrm{Mem} = f((\mathbf{k}_i, \mathbf{v}_i), \mathrm{Mem}_{\mathrm{pre}})

0

1

Updated 2026-04-22

Contributors are:

Who are from:

Tags

Foundations of Large Language Models

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences