Some tasks are not well-suited to content-based addressing. In certain cases, even though the content of a variable is arbitrary, variables still require recognizable names and addressing. Arithmetic problems provide a good example; variables such as $$x$$ and $$y$$ can take any values, but we still need to define a function formally, such as $$f(x, y) = x * y$$.

San Diego State University

Google

In the Neural Turing Machine (NTM) architecture, content-based addressing focuses on memory locations based on the similarity between values emitted by the controller and the contents of memory. For both read and write operations, the controller first emits a key vector of length $$M$$, denoted as $$k_t$$. This key vector is compared to each memory vector, $$M_t(i)$$.

This method produces normalized weights, $$w_t^c(i)$$, using a positive key strength, $$\beta_t$$:
$$w_t^c(i) = \frac {\exp(\beta_t K[k_t, M_t(i)])} {\sum_j \exp(\beta_t K[k_t, M_t(j)])}$$

The similarity is measured using cosine similarity:
$$K[u, v] = \frac {u \cdot v} {\|u\| \|v\|}$$

Learn Before

Related