Teacher forcing is a method for quickly and efficiently training recurrent neural network models that use the ground truth from a prior time step as input.

University of Michigan - Ann Arbor

In an RNN, backpropagation needs to take into account that the model is carrying forward information from each neural layer to the next, and fine tune the weights that govern this “short-term memory”.

This is called Backpropagation Through Time (BPTT). BPTT uses the chain rule to go back from the latest time step to the previous step, and then to the next-previous, each time using gradient descent to discover the best weights for each neuron and for the hidden state function.

Backpropagation Through Time (BPTT)

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

In a neural network with many time steps or layers, a gradient at the early layer is the product of all the terms from the later layers, which leads to an inherently unstable situation. Especially when the value of gradient has become so small, it no longer updates properly or is vanished eventually. Exploding gradient can be considered as the opposite of vanishing process. The updated weights using gradient descent become so large that they cause the whole network to become unstable, which leads to numerical overflow. 

Vanishing/exploding gradient

Let's start with a quick recap of the basic equations in RNN:
$$h^{(t)}=g_1(Wh^{(t-1)} \oplus Ux^{(t)}+b_h)$$
$$\hat y^{(t)}=g_2(Vh^{(t)}+b_y)$$
$$J^{(t)} = \mathcal{L} (\hat{y}^{(t)}, y^{(t)}) ~~~~~and~~~~~ J = \frac{1}{T} \sum_{t=1}^T J^{(t)} $$

We want to calculate the gradients of the loss function with parameters U,V, and W. We can sum up the gradients at each step since $\frac{\partial J}{ \partial W} = \frac{1}{T} \sum_{t=1}^T \frac{\partial J^{(t)}}{\partial W} $. Thus we only need to find $\frac{\partial J^{(t)}}{\partial W} $, $\frac{\partial J^{(t)}}{\partial V} $, and $\frac{\partial J^{(t)}}{\partial U} $.

Derivative of V only depends on the current step:
$$\frac{\partial J^{(t)}}{\partial V}  = \frac{\partial J^{(t)} }{\partial \hat{y}^{(t)}} \frac{\partial \hat{y}^{(t)}}{\partial V} $$

But it is not the case for derivative of U and W, take t=3 as an example:
$$\frac{\partial J^{(3)}}{\partial W}  = \sum_{k=1}^3 \frac{\partial J^{(3)} }{\partial \hat{y}^{(3)}} \frac{\partial \hat{y}^{(3)}}{\partial h^{(3)}} \frac{\partial h^{(3)}}{\partial h^{(k)}} \frac{\partial h^{(k)}}{\partial W}$$
$$ =\sum_{k=1}^3 \frac{\partial J^{(3)} }{\partial \hat{y}^{(3)}} \frac{\partial \hat{y}^{(3)}}{\partial h^{(3)}} (\prod_{j=k+1}^3 \frac{\partial h^{(j)}}{\partial h^{(j-1)}}) \frac{\partial h^{(k)}}{\partial W}$$
and
$$\frac{\partial J^{(3)}}{\partial U}  = \sum_{k=1}^3 \frac{\partial J^{(3)} }{\partial \hat{y}^{(3)}} \frac{\partial \hat{y}^{(3)}}{\partial h^{(3)}} \frac{\partial h^{(3)}}{\partial h^{(k)}} \frac{\partial h^{(k)}}{\partial U}$$
$$ =\sum_{k=1}^3 \frac{\partial J^{(3)} }{\partial \hat{y}^{(3)}} \frac{\partial \hat{y}^{(3)}}{\partial h^{(3)}} (\prod_{j=k+1}^3 \frac{\partial h^{(j)}}{\partial h^{(j-1)}}) \frac{\partial h^{(k)}}{\partial U}$$

RNN Backpropagation Formulas

A helpful website for understanding backpropagation in RNN:
http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/

Helpful Website for BPTT

Teacher forcing

Weight-tying is where you have a language model and use the same weight matrix for the input-to-embedding layer (the input embedding) and the hidden-to-softmax layer (the output embedding). The idea is that these two matrices contain essentially the same information, each having a row per word in the vocabulary.

In addition to providing improved perplexity results, this approach significantly reduces the number of parameters required for the model.

Learn Before

Related