If the variables are continuous, a linear auto-regressive model is merely another
way to formulate a multivariate Gaussian distribution.

University of Michigan - Ann Arbor

The simplest form of auto-regressive network has no hidden units and no sharing
of parameters or features. And each conditional probability is parametrized as a linear
model.

Linear Auto-Regressive Network

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

A special degenerating condition of linear ARN

They may be trained with convex loss functions and sometimes admit closed form solutions

Advantages of Linear ARN

The model itself does not offer a way of increasing its capacity, so capacity must be raised using techniques like basis expansions of the input or the kernel trick.

Disadvantages of Linear ARN

Neural Auto-Regressive Networks

A linear auto-regressive network has $$O(d^2)$$ parameters when modeling $$d$$ variables, because each of the $$d$$ conditional probabilities is parameterized as a linear model with no parameter sharing.

Learn Before

Related