Norms, which quantify the magnitude of a vector, are commonly applied to compute the distance between two vectors. This is achieved by calculating the norm of the difference between the two vectors, which mathematically measures how far apart they are in the vector space. For example, the distance between vectors $$\mathbf{u}$$ and $$\mathbf{v}$$ is computed as $$\lVert\mathbf{u} - \mathbf{v}\rVert$$.

Claude

A norm is a mathematical function $$\| \cdot \|$$ that maps a vector to a scalar, informally measuring its overall size or magnitude. To be a valid norm, the function must satisfy three properties for any vectors $$\mathbf{x}$$ and $$\mathbf{y}$$ and scalar $$\alpha \in \mathbb{R}$$:

1. Scaled norm: $$\left\|\alpha \mathbf{x}ight\| = |\alpha| \left\|\mathbf{x}ight\|$$
2. Triangle inequality: $$\left\|\mathbf{x} + \mathbf{y}ight\| \leq \left\|\mathbf{x}ight\| + \left\|\mathbf{y}ight\|$$
3. Nonnegativity: $$\left\|\mathbf{x}ight\| > 0$$ for all $$\mathbf{x} 
eq 0$$.

Vector Norm

Dive into Deep Learning

In deep learning, distances and sizes are frequently used to formulate optimization objectives, and these are often mathematically expressed as norms. For example, norms are used to minimize the distance between predictions and ground truth observations, or to maximize the distance between representations of distinct entities (like photos of different people).

Norms as Objectives in Deep Learning

The $$\ell_2$$ norm, also known as the Euclidean norm, measures the Euclidean length of a vector. It is calculated as the square root of the sum of the squares of the vector's elements. For a vector $$\mathbf{x}$$ with $$n$$ elements, the $$\ell_2$$ norm is formally expressed as:
$$\left\|\mathbf{x}\right\|_2 = \sqrt{\sum_{i=1}^n x_i^2}$$

Learn Before

Related