Due to the similarities between a NADE network and an RBM, one can perform mean field inferences on the NADE system in a computationally similar way. However, as the RBM is a recurrent network, only one inference can be made at a time. This is in contrast to the discrete nature of a NADE network, which allows it to perform a finite number of inferences $k$ simultaneously, where $k$ is less than the total number of inputs the NADE network was trained to handle.

NADE-k

NADE or a Neural Auto-regressive Density Estimator is largely identical to a standard Neural Auto-Regressive Network, but with one slight difference.

Input values $x_i$ are supplied into the network only after being weighted by weight vectors $W'_{j,k,i} = W_{k,i}$

Or the weight vector $W_{k,i}$ is reused for all inputs into hidden unit groups $h^{(j)}$ where $j>=i$

This allows for forward propagation of a NADE model to somewhat resemble the computation of missing inputs in a mean field inference for an RBM (Restricted Boltzmann Machine).

University of Toledo

Neural Auto-Regressive Networks

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Learn Before

Related

Learn After