It introduces the relationship of Higher-order variables with fewer unique parameters and less computation.

University of Michigan - Ann Arbor

Auto-Regressive Networks are a kind of directed probability graph. The conditional probability is described using neural networks and the chain rule. The key definition of an Auto-Regressive Network is: $$p(x) =\prod_{i=1}^{n_i} p(x_i|x_{i-1},...,x_1)$$

Auto-Regressive Network

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

Advantages of ARN

The simplest form of auto-regressive network has no hidden units and no sharing
of parameters or features. And each conditional probability is parametrized as a linear
model.

Linear Auto-Regressive Network

If an auto-regressive network has been trained on predicting values when given input in an arbitrary order, there is a simple technique that can be used to improve the accuracy of a prediction. Given $$n$$ input vectors, there are $$n!$$ many ways to arrange those vectors in order to predict the next output. Let $$n! \geq k$$ and let $$o^{(i)}$$ be a unique input permutation for $$i$$ ranging from 1 to $$k$$. $$p_{ensemble}(x) = \frac{1}{k}\sum_{i=1}^{k} p(x | o^{(i)})$$

Learn Before

Related