The concept of using more than one RNN that have some type of relationship between the hidden units is expandable to any number of dimensions of inputs you may have. For example, in our 1 dimensional example of "Bidirectional RNNs", the proposal was to use two networks with hidden units feeding into each other (one net for the past and one for the future).

This concept can be used with any $n$ dimensional input model. In image recognition, four recursive neural networks can be used, two for the vertical axis and two for the horizontal axis.

University of Connecticut

A bidirectional RNN assumes that the correct output not only depends on the previous inputs in the time series but also on future inputs. For example, in translation models, it is often necessary to use a word from the end of the source sentence in order to predict a word early in the target sentence. 

To make this possible, two RNNs are stacked on top of each other, one going from beginning to end and the other from end to beginning, and the output is computed based on hidden states of both networks.

Bidirectional RNNs

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

A deep RNN uses multiple neuron layers per time step. This is, in essence, a three-dimensional neural network which progresses from left to right, from right to left, and from bottom to top. It doesn't matter if the model performs forward propagation from left to right or from right to left, but it needs to propagate information through time first and then through layers. This architecture provides higher learning capacity, but also requires more training data.  

Deep (Bidirectional) RNNs

An ordinary RNN is useful because the model takes in information about your output at a previous time step to predict your current and future outputs. Similarly, a Bidirectional RNN depends upon both outputs at previous and FUTURE outputs. This is useful in applications where your predictions in your model not only depend on the past, but the future. This would include things like handwriting recognition, speech recognition, and bioinformatics. 

Learn Before

Related