Activity (Process)

Getting Attention Weight in DKVMN

In the Dynamic Key-Value Memory Network (DKVMN), the first step of its working mechanism is getting the attention weight. The knowledge component qtq_t is extracted from the embedding matrix and used as a query ktk_t to search the key memory matrix Mk\mathbf{M}^k. This produces an attention weight vector that measures how much attention should be paid to each value in the value memory matrix:

wti=Softmax(Mikkt)w_{ti} = \text{Softmax}(\mathbf{M}_i^k k_t)

where i=1Nwti=1\sum_{i=1}^N w_{ti} = 1, Mik\mathbf{M}_i^k is the ii-th row vector of the key memory, and wtiw_{ti} is the ii-th element of the attention weight vector.

0

1

Updated 2026-06-16

Contributors are:

Who are from:

Tags

Data Science