It introduces the relationship of Higher-order variables with fewer unique parameters and less computation.

University of Michigan - Ann Arbor

Auto-Regressive Networks are a kind of directed probability graph. The conditional probability is described using neural network and chain rule. The key defination of the  Auto-Regressive Network is:
$p(x) =\prod_{i=1}^{n_i} p(x_i|x_{i-1},...,x_1)$

Auto-Regressive Network

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

Advantages of ARN

The simplest form of auto-regressive network has no hidden units and no sharing
of parameters or features. And each conditional probability is parametrized as a linear
model.

Linear Auto-Regressive Network

If an auto-regressive network has been trained on predicting values when given input in an arbitrary order, there is a simple technique that can be used to improve the accuracy of a prediction.

Given $n$ input vectors, there are $n!$ many ways to arrange those vectors in order to predict the next output. Let $n! \geq k$ and let $o^{(i)}$ be a unique input permutation for $i$ ranging from 1 to $k$. 

$$p_{ensemble}(x) = \frac{1}{k}\sum_{i=1}^{k} p(x | o^{(i)})$$ 


Learn Before

Related