The $$\ell_2$$ norm of a vector can be calculated programmatically using linear algebra utilities in modern deep learning frameworks. In PyTorch and TensorFlow, this is accomplished by passing the vector to `torch.norm(u)` and `tf.norm(u)`, respectively. In JAX and NumPy (used by MXNet), it is computed using the linear algebra module via `jnp.linalg.norm(u)` and `np.linalg.norm(u)`.

Programmatic L2 Norm Calculation

The $$\ell_2$$ norm, also known as the Euclidean norm, measures the Euclidean length of a vector. It is calculated as the square root of the sum of the squares of the vector's elements. For a vector $$\mathbf{x}$$ with $$n$$ elements, the $$\ell_2$$ norm is formally expressed as:
$$\left\|\mathbf{x}\right\|_2 = \sqrt{\sum_{i=1}^n x_i^2}$$

Claude

A norm is a mathematical function $$\| \cdot \|$$ that maps a vector to a scalar, informally measuring its overall size or magnitude. To be a valid norm, the function must satisfy three properties for any vectors $$\mathbf{x}$$ and $$\mathbf{y}$$ and scalar $$\alpha \in \mathbb{R}$$:

1. Scaled norm: $$\left\|\alpha \mathbf{x}ight\| = |\alpha| \left\|\mathbf{x}ight\|$$
2. Triangle inequality: $$\left\|\mathbf{x} + \mathbf{y}ight\| \leq \left\|\mathbf{x}ight\| + \left\|\mathbf{y}ight\|$$
3. Nonnegativity: $$\left\|\mathbf{x}ight\| > 0$$ for all $$\mathbf{x} 
eq 0$$.

Vector Norm

Dive into Deep Learning

In deep learning, distances and sizes are frequently used to formulate optimization objectives, and these are often mathematically expressed as norms. For example, norms are used to minimize the distance between predictions and ground truth observations, or to maximize the distance between representations of distinct entities (like photos of different people).

Norms as Objectives in Deep Learning

L2 Norm

Norms, which quantify the magnitude of a vector, are commonly applied to compute the distance between two vectors. This is achieved by calculating the norm of the difference between the two vectors, which mathematically measures how far apart they are in the vector space. For example, the distance between vectors $$\mathbf{u}$$ and $$\mathbf{v}$$ is computed as $$\lVert\mathbf{u} - \mathbf{v}\rVert$$.

Learn Before

Related

Learn After