Learn Before
Concept

Process of Annealed Importance Sampling

A series of Markov chain transition functions are applied to define the conditional probability distribution. By chaining the importance weights for the jumps between the distributions throughout the sampling process, we can derive the importance weight, and estimate the ratio of partition functions:

w(k)=p~η1(xη1(k))p~0(xη1(k))p~η2(xη2(k))p~η1(xη2(k))p~1(x1(k))p~ηn1(xηn(k))w^{(k)}=\frac{\tilde{p}_{\eta_{1}}\left(\boldsymbol{x}_{\eta_{1}}^{(k)}\right)}{\tilde{p}_{0}\left(\boldsymbol{x}_{\eta_{1}}^{(k)}\right)} \frac{\tilde{p}_{\eta_{2}}\left(\boldsymbol{x}_{\eta_{2}}^{(k)}\right)}{\tilde{p}_{\eta_{1}}\left(\boldsymbol{x}_{\eta_{2}}^{(k)}\right)} \cdots \frac{\tilde{p}_{1}\left(\boldsymbol{x}_{1}^{(k)}\right)}{\tilde{p}_{\eta_{n-1}}\left(\boldsymbol{x}_{\eta_{n}}^{(k)}\right)}

Z1Z01Kk=1Kw(k)\frac{Z_{1}}{Z_{0}} \approx \frac{1}{K} \sum_{k=1}^{K} w^{(k)}

0

1

Updated 2026-07-01

References


Tags

Deep Learning

Data Science

Machine Learning Yearning @ DeepLearning.AI

Dive into Deep Learning @ D2L

Machine Learning

Supervised Learning