$O(d^2)$ parameters when there are d variables to model.

University of Michigan - Ann Arbor

The simplest form of auto-regressive network has no hidden units and no sharing
of parameters or features. And each conditional probability is parametrized as a linear
model.

Linear Auto-Regressive Network

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

If the variables are continuous, a linear auto-regressive model is merely another
way to formulate a multivariate Gaussian distribution.

A special degenerating condition of linear ARN

Magnitude of Parameters of Linear ARN

They may be trained with convex loss functions and sometimes admit closed form solutions

Advantages of Linear ARN

The model itself does not offer a way of increasing its capacity, so capacity must be raised using techniques like basis expansions of the input or the kernel trick.

Learn Before

Related