Learn Before
Formula
Concat Attention Scoring Function
The concat (concatenation) attention scoring function is one of the three attention scoring functions proposed by Luong et al. (2015). It calculates the alignment score between an encoder hidden state and a decoder hidden state as:
where:
- is the encoder vector (source hidden state).
- is the decoder vector (target hidden state at time step ).
- and are learnable weight parameters.
- represents the concatenation of the encoder and decoder states.
In a neural network, this is implemented by concatenating the encoder and decoder states, applying a dense layer with a activation, and projecting the result to a single scalar to produce the final score. During training, this mechanism learns which source words are most influential for generating target words.
0
1
Updated 2026-06-18
Contributors are:
Who are from:
Tags
Data Science