Formula

RNN Decoder Hidden State Recurrence

Within a sequence-to-sequence decoder, the RNN updates its hidden state at each time step tt' using a transformation function gg that takes three inputs: the previous target token yt1y_{t'-1}, the encoder's context variable c\mathbf{c}, and the decoder's hidden state from the preceding time step st1\mathbf{s}_{t'-1}. The resulting hidden state update is expressed as:

st=g(yt1,c,st1)\mathbf{s}_{t'} = g(y_{t'-1}, \mathbf{c}, \mathbf{s}_{t'-1})

This formulation mirrors the encoder's recurrence but differs in a key way: the decoder's transformation incorporates the context variable c\mathbf{c} as an additional input at every time step, ensuring that the encoded source sequence information continuously influences the generation process.

0

1

Updated 2026-05-14

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L