Due to the similarities between a Neural Autoregressive Density Estimator (NADE) and a Restricted Boltzmann Machine (RBM), one can perform mean field inferences on the NADE system in a computationally similar way. However, as the RBM is a recurrent network, only one inference can be made at a time. This is in contrast to the discrete nature of a NADE network, which allows it to perform a finite number of inferences $$k$$ simultaneously, where $$k$$ is less than the total number of inputs the NADE network was trained to handle.

NADE-k

A Neural Autoregressive Density Estimator (NADE) is largely identical to a standard Neural Auto-Regressive Network, but with one key difference in parameter sharing. Input values $$x_i$$ are supplied into the network only after being weighted by weight vectors $$W'_{j,k,i} = W_{k,i}$$. In other words, the weight vector $$W_{k,i}$$ is reused for all inputs into hidden unit groups $$h^{(j)}$$ where $$j \geq i$$. This allows the forward propagation of a NADE model to resemble the computation of missing inputs in a mean field inference for a Restricted Boltzmann Machine (RBM).

University of Toledo

Google

Neural Auto-Regressive Networks

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Learn Before

Related

Learn After