Learn Before
Concept

LSTM Forward Pass Equations

The forget gate is Gamma_f=sigma(W_f[a^{}, x^{}]+b_f), where \sigma denotes the sigmoid activation function, W_f is the weight matrix, b_f is a bias term, a^{} denotes the hidden state from the previous time step, and x^{} is the input at the t-th time step. The notation [a^{}, x^{}] means that a^{} and x^{} are concatenated. Then, compute the update gate in two steps. First, the update gate is Gamma_u=sigma(W_u[a^{}, x^{}]+b_u). Second, the intermediate cell state candidate is tilde{c}^{}=tanh(W_c[a^{}, x^{}]+b_c), where \tanh denotes the hyperbolic tangent activation function. Using the results from the formulas above, we can calculate the current cell state as c^{}=\Gamma_u*\tilde{c}^{}+\Gamma_fc^{}. Finally, the third gate, the output gate, is Gamma_o=sigma(W_o[a^{}, x^{}]+b_o), and using the output gate and the current cell state, we can compute the current hidden state as a^{}=\Gamma_o\tanh(c^{}).

Image 0

0

1

Updated 2026-06-14

Tags

Data Science