A norm maps each vector of a vector space to a
nonnegative real number that measures its length. It is definite,
homogeneous, and satisfies the triangle inequality. Every norm
induces a metric, and the inner product of a
Hilbert space induces a norm. In machine learning (ML), the family of
$\ell_p$-norms is used to define loss functions and
regularizers, e.g., the $\ell_1$-norm of least absolute shrinkage and selection operator (Lasso).
Definition
A norm is a function that maps each (vector) element
of a vector space to a nonnegative real number. Formally, a norm
$\normgeneric{\, \cdot \,}{}$ on a vector space $\vecspace$ over a field $\mathbb{F}$ is a
function $\normgeneric{\, \cdot \,}{}: \vecspace \to \mathbb{R}_{+}$
that satisfies the following conditions (Horn and Johnson, 2013) for all $\vu, \vv \in \vecspace$
and $\expcoeff \in \mathbb{F}$:
Intuitively, norms measure the length or size of a vector.
Norms can also be used to measure distances between vectors, as they define a metric by
$\metric{\vu}{\vv} \defeq \normgeneric{\vu - \vv}{}$,
making $\pair{\vecspace}{\metric{\cdot}{\cdot}}$ a metric space.
On an inner product space, the inner product induces a
norm via $\normgeneric{\vu}{} \defeq \sqrt{\innerprod{\vu}{\vu}}$;
this is the norm of a Hilbert space
(see inner product, Hilbert space).
A prominent family of norms used in machine learning (ML) is the $\ell_p$-norms for $p \geq 1$,
defined for a vector $\featurevec \in \mathbb{R}^\featuredim$ as
$\normgeneric{\featurevec}{p} = \left( \displaystyle \sum_{\featureidx=1}^{\featuredim} |\feature_{\featureidx}|^p \right)^{1/p}$.
Important instances include the $\ell_1$-norm, the $\ell_2$-norm (or Euclidean norm),
and the $\ell_\infty$-norm $\normgeneric{\featurevec}{\infty} = \max_{\featureidx = 1, \ldots, \featuredim} |\feature_{\featureidx}|$, which are illustrated
by the geometry of their respective unit spheres in Fig.\ 1.
In ML, norms are fundamental, for instance for defining a loss function or a regularizer.
Figure 1: Unit spheres $\sphere{1}$ with respect to different $\ell_p$-norms for $p=1,\,2,\,\infty$
See also: vector space, metric, metric space,
Euclidean space, Hilbert space, inner product,
linear regression, least absolute shrinkage and selection operator (Lasso).
References
Horn and Johnson (2013). Matrix Analysis. Cambridge Univ. Press.
Cite this entry
@misc{dictml_norm,
author = {Jung, Alexander},
title = {norm},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-08-06},
url = {https://dictionaryofml.org/terms/norm.html}
}
