Considering Boltzmann machine as a set of units $x$ with a weight matrix $U$ and biases $b$, the energy function is given by: 
$$E(x) = -x^{T}Ux-b^{T}x$$
we can implement structures of Boltzmann machines with diﬀerent numbers of layers, by partitioning $x$ into visible and hidden units and zeroing out elements of $U$ for units that do not interact.
$$E^{'}(x;U,b) = -(x-\mu)^{T}U(x-\mu)-(x-\mu)^{T}b$$
Typically $\mu$ is a hyperparameter ﬁxed at the beginning of training. It is usually chosen to make sure that $x-\mu \approx 0$ when the model is initialized. 


State University of New York at Stony Brook

Centered deep Boltzmann machine reparametrizes the model to make the Hessian of the cost function better conditioned at the beginning of the learning process. This yields a model that can be trained without a greedy layer-wise pretraining stage.  The resulting model obtains excellent test set log-likelihood and produces high-quality samples. Unfortunately, it remains unable to compete with appropriately regularized MLPs as a classiﬁer.




Centered Deep Boltzmann Machine

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Learn Before

Related