The $$L_1$$ norm, also known as the Manhattan norm or Taxicab norm, measures the sum of the absolute values of a vector's elements. For a vector $$\mathbf{x} \in \mathbb{R}^n$$, the $$L_1$$ norm is defined as:
$$\|\mathbf{x}\|_1 = \sum_{i=1}^n |x_i|$$

L1 Norm

The $$\ell_2$$ norm, also known as the Euclidean norm, measures the Euclidean length of a vector. It is calculated as the square root of the sum of the squares of the vector's elements. For a vector $$\mathbf{x}$$ with $$n$$ elements, the $$\ell_2$$ norm is formally expressed as:
$$\left\|\mathbf{x}\right\|_2 = \sqrt{\sum_{i=1}^n x_i^2}$$

L2 Norm

The max norm, also known as the $$L^\infty$$ norm, measures the absolute value of the largest element in a vector. It is a special case of the $$L^p$$ norm, obtained in the limit as $$p \to \infty$$. For a vector $$\mathbf{x} \in \mathbb{R}^n$$, the max norm is formally defined as: $$\left\|\mathbf{x}\right\|_\infty = \max_{i} |x_i|$$

Max Norm

The $$L^p$$ norm (or $$\ell_p$$ norm) is a mathematical function that measures the size or length of a vector. For a vector $$\mathbf{x} \in \mathbb{R}^n$$ and a real number $$p \geq 1$$, the $$L^p$$ norm is defined as: $$\left\|\mathbf{x}\right\|_p = \left( \sum_{i=1}^n |x_i|^p \right)^{1/p}$$ Common vector norms are special cases of the $$L^p$$ norm: - The $$L_1$$ norm is obtained by setting $$p=1$$. - The $$L_2$$ norm (Euclidean norm) is obtained by setting $$p=2$$. - The max norm ($$L^\infty$$ norm) is obtained in the limit as $$p \to \infty$$, resulting in: $$\left\|\mathbf{x}\right\|_\infty = \max_{i} |x_i|$$

University of Minnesota - Twin Cities

Google

A norm is a mathematical function $$\| \cdot \|$$ that maps a vector to a scalar, informally measuring its overall size or magnitude. To be a valid norm, the function must satisfy three properties for any vectors $$\mathbf{x}$$ and $$\mathbf{y}$$ and scalar $$\alpha \in \mathbb{R}$$:

1. Scaled norm: $$\left\|\alpha \mathbf{x}ight\| = |\alpha| \left\|\mathbf{x}ight\|$$
2. Triangle inequality: $$\left\|\mathbf{x} + \mathbf{y}ight\| \leq \left\|\mathbf{x}ight\| + \left\|\mathbf{y}ight\|$$
3. Nonnegativity: $$\left\|\mathbf{x}ight\| > 0$$ for all $$\mathbf{x} 
eq 0$$.

Vector Norm

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

In deep learning, distances and sizes are frequently used to formulate optimization objectives, and these are often mathematically expressed as norms. For example, norms are used to minimize the distance between predictions and ground truth observations, or to maximize the distance between representations of distinct entities (like photos of different people).

Norms as Objectives in Deep Learning

Norms, which quantify the magnitude of a vector, are commonly applied to compute the distance between two vectors. This is achieved by calculating the norm of the difference between the two vectors, which mathematically measures how far apart they are in the vector space. For example, the distance between vectors $$\mathbf{u}$$ and $$\mathbf{v}$$ is computed as $$\lVert\mathbf{u} - \mathbf{v}\rVert$$.

Learn Before

Related

Learn After