Learn Before
Relation
Probability Distribution and Generative Learning
- On the forward pass, an RBM uses input values to compute the hidden layer's activations (its predictions). - On the backward pass, those hidden activations are fed back through the same weights to produce a reconstruction: an estimate of . - Because this reconstruction estimates a probability distribution over the input rather than a single deterministic value, the process is called generative learning. - An RBM compares its estimated distribution to the true input distribution using Kullback-Leibler (KL) Divergence, which measures the non-overlapping area between the two distributions. - Training minimizes this KL Divergence, adjusting the shared weights so that the reconstruction becomes an increasingly close approximation of the original input.
0
1
Updated 2026-07-11
Contributors are:
Who are from:
Tags
Data Science