The RNN-RBM sequence model is a generative model of a sequence of frames $\boldsymbol x^{(t)}$ consisting of an RNN that emits RBM parameters for each time step. This model differs from other approaches because it uses the RNN to emit all of the parametrs of the RBM rather than just varying the bias parameters for each time step. The model is trained by back-propagating the gradient of the loss function through the RNN. The loss function is not applied directly to the RNN outputs, but it is applied to the RBM.

RNN-RBM Sequence Models

Boltzmann machines can also be used for sequence modelling tasks using the same concept of conditional modeling. The difference lies in the fact that in sequence modeling rather than mapping an input $\boldsymbol x$ to an ouptut $\boldsymbol y$ , the model must represent factors of the form $p(\textbf{x}^{(\tau)}| \textbf{x}^{(1)}, \dotsc, \textbf{x}^{(\tau-1)})$ in order to estimate a probability distribution over a sequence of variables, $p(\textbf{x}^{(1)}, \dotsc,  \textbf{x}^{(\tau)})$

New York University

If a scenario requires the modelling of some output $\boldsymbol y$, where the different entries of $\boldsymbol y$ are related to each other and must obey certain constraints, from some input $\boldsymbol x$, we have what is called a structured output scenario. For example, $\boldsymbol y$ can be a waveform in a speech synthesis task which must sound coherent. The natural way to represent the relationships between the entries in $\boldsymbol y$ is to extend a Boltzmann machine to model conditional distributions of the form $p(\mathbf{y}| \boldsymbol x)$.

Boltzmann Machines for Structured Outputs

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Learn Before

Related

Learn After