Software implementations of DBMs need to have many diﬀerent components for CD training of individual RBMs, PCD training of the full DBM, and training based on back-propagation through the MLP. The MLP on top of the Boltzmann machine loses many of the advantages of the Boltzmann machine probabilistic model, such as being able to perform inference when some input values are missing. 

There are two main ways to resolve the joint training problem of the deep Boltzmann machine, centered deep Boltzmann machine and multi-prediction deep Boltzmann machine


Jointly Training Deep Boltzmann Machines

Deep Boltzmann Machine (DBM) is a deep generative model. Unlike Restricted Boltzmann Machine, DBM has multiple layers of latent variable. In each layer, each of the latent variable is mutually independent, and conditioned on their neighboring layers. This property allows us to use fixed-point equations and find the true optimal mean field expectation. The use of mean field simulates human brain connections and DBM is therefore used in neuroscientific phenomena. One undesired property of DBM is that sampling is difficult.

Johns Hopkins University

The deep learning approach is to figure out the minimum amount of information that is absolutely needed and find out how to get a reasonable approximation of this information in the shortest amount of time. 

The latent variables are designed differently in which the training algorithm can invent the concepts it needs to model a particular dataset freely and deal with complex problems and are reusable in more different contexts.


The Deep Learning Approach to Structured Probabilistic Models

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

-	Restricted Boltzmann Machine is an undirected graphical model that plays a major role in Deep Learning Framework in recent times. It was initially introduced as Harmonium by Paul Smolensky in 1986 and it gained big popularity in recent years in the context of the Netflix Prize where Restricted Boltzmann Machines achieved state of the art performance in collaborative filtering and have beaten most of the competition.


Restricted Boltzmann Machine

Deep Boltzmann Machine

Deep graphical models have large groups of units that are all connected to other groups of units. They tend to connect each visible unit $v_i$ to many hidden units $h_i$ for $h$ to provide a distributed representation of $v_i$

Learn Before

Related

Learn After