Learn Before
Concept

Contrastive Divergence

In order to improve upon the naive implementation of Monte Carlo methods to approvimately maximise lieklihoods of models, the contrastive divergence algorithm is employed. The difference between the naive implementation and the contrastive divergence algorithm lies in the fact that the contrastive divergence algorithm initializes the Markov chain at each step with data samples rather than randomized values. This solves a major inefficiency where the burn in operation does not take as many steps because the Markov chains are initialized from samples of the data generating distribution which is much closer to the model distribution than randomised sampling. Contrastive divergence remains an approximation to the correct negative phase and hence is not perfect; most notably, it fails to suppress regions of high probability that are far from actual training examples.

0

1

Updated 2021-07-22

References


Tags

Data Science

Related