Centered deep Boltzmann machine reparametrizes the model to make the Hessian of the cost function better conditioned at the beginning of the learning process. This yields a model that can be trained without a greedy layer-wise pretraining stage. The resulting model obtains excellent test set log-likelihood and produces high-quality samples. Unfortunately, it remains unable to compete with appropriately regularized MLPs as a classifier.