If the variables are continuous, a linear auto-regressive model is merely another
way to formulate a multivariate Gaussian distribution.

A special degenerating condition of linear ARN

$O(d^2)$ parameters when there are d variables to model.

Magnitude of Parameters of Linear ARN

They may be trained with convex loss functions and sometimes admit closed form solutions

Advantages of Linear ARN

The model itself does not offer a way of increasing its capacity, so capacity must be raised using techniques like basis expansions of the input or the kernel trick.

Disadvantages of Linear ARN

Neural Auto-Regressive Networks

The simplest form of auto-regressive network has no hidden units and no sharing
of parameters or features. And each conditional probability is parametrized as a linear
model.

University of Michigan - Ann Arbor

Auto-Regressive Networks are a kind of directed probability graph. The conditional probability is described using neural network and chain rule. The key defination of the  Auto-Regressive Network is:
$p(x) =\prod_{i=1}^{n_i} p(x_i|x_{i-1},...,x_1)$

Auto-Regressive Network

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

It introduces the relationship of Higher-order variables with fewer unique parameters and less computation.

Advantages of ARN

Linear Auto-Regressive Network

If an auto-regressive network has been trained on predicting values when given input in an arbitrary order, there is a simple technique that can be used to improve the accuracy of a prediction.

Given $n$ input vectors, there are $n!$ many ways to arrange those vectors in order to predict the next output. Let $n! \geq k$ and let $o^{(i)}$ be a unique input permutation for $i$ ranging from 1 to $k$. 

$$p_{ensemble}(x) = \frac{1}{k}\sum_{i=1}^{k} p(x | o^{(i)})$$ 


Learn Before

Related

Learn After