logo
How it worksCoursesResearch CommunitiesBenefitsAbout Us
Schedule Demo
Learn Before
Relation

Algorithm of Restricted Boltzmann Machine

  • The canonical RBM is an energy-based model with binary visible and hidden units.
  • Its energy function is E(v,h)=−bTv−cTh−vTWhE(v,h) = -b^Tv - c^Th - v^TWh, where b, c, and W are unconstrained, real valued, learnable parameters.
  • It is easy to take its derivatives: ∂∂Wi,jE(v,h)=−vihj\frac{\partial}{\partial W_{i,j}}E(v,h) = -v_ih_j.
  • There are no direct interactions between any two visible units or any two hidden units (restrictions).
  • The restrictions on the RBM structure yield the nice properties p(h∣v)=∏ip(hi∣v)p(h|v)=\prod_ip(h_i|v) and p(v∣h)=∏ip(vi∣h)p(v|h) = \prod_ip(v_i|h).
  • The individual conditionals are simple to compute. For the binary RBM we obtain: P(hi=1∣v)=ρ(vTW:,i+bi)P(h_i=1|v) = \rho(v^TW_{:,i} + b_i), and P(hi=1∣v)=1−ρ(vTW:,i+bi)P(h_i=1|v) =1 - \rho(v^TW_{:,i} + b_i).
  • Together, these properties allow for efficient block Gibbs sampling and efficient derivatives - make training convenient. Samples generated by Gibbs sampling from an RBM model are shown below:
Image 0

0

1

Updated 2021-07-14

Contributors are:

XZ

Xinyi Zhao
🏆 6

Who are from:

University of Michigan - Ann Arbor
University of Michigan - Ann Arbor
🏆 6

Tags

Data Science

Related
Learn After