In the cross-attention layer of the transformer implementation for the encoder-decoder architecture, the final output of the encoder $H^{enc}$ is multiplied by the cross-attention layer’s key weights $W^K$ and value weights $W^V$, but the output from the prior decoder layer $H^{dec[i-1]}$ us multiplied by cross-attention layer’s query weights $W^Q$: $Q = W^QH^{dec[i-1]}; \; K = W^KH^{enc}; \; V = W^VH^{enc}$ $\implies \text{CrossAttention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$ where $d_k$ is the dimension of the key vector.