projection

Typeset PDF version — the authoritative form of this entry

The projection of a vector onto a closed non-empty subset of a Euclidean space is a point in the subset that is closest to the vector in the Euclidean norm. If the subset is convex, this closest point is unique. If the subset is a subspace, the projection is a linear map, namely the orthogonal projection onto the subspace. In machine learning (ML), projections enforce constraints on model parameters during training: projected gradient descent (projected GD) alternates gradient steps with projections onto the constraint set, e.g., the $\ell_{1}$-ball in an equivalent formulation of the least absolute shrinkage and selection operator (Lasso).

Definition

Consider a closed non-empty subset $\paramspace \subseteq \reals^{\dimlocalmodel}$ of the $\dimlocalmodel$-dimensional Euclidean space. The projection $\projection{\paramspace}{\weights}$ of a vector $\weights \in \reals^{\dimlocalmodel}$ onto $\paramspace$ is defined as \begin{equation} \nonumber \label{equ_def_proj_generic_dict} \projection{\paramspace}{\weights} = \argmin_{\weights' \in \paramspace} \normgeneric{\weights - \weights'}{2}\text{.} \end{equation} In other words, $\projection{\paramspace}{\weights}$ is a vector in $\paramspace$ that is closest to $\weights$. The above minimum exists for every closed non-empty subset $\paramspace$; if $\paramspace$ is also convex, the closest point is unique (Boyd and Vandenberghe, 2004). If $\paramspace$ is a subspace, the map $\weights \mapsto \projection{\paramspace}{\weights}$ is linear: it is the orthogonal projection onto $\paramspace$.

In machine learning (ML), projections are used to enforce constraints on model parameters during training. As a case in point, consider linear regression with the constraint $\normgeneric{\weights}{1} \leq \tau$, which is an equivalent formulation of the least absolute shrinkage and selection operator (Lasso) (Tibshirani, 1996). This constrained problem can be solved by projected gradient descent (projected GD): each iteration takes a gradient step and then projects the result onto the $\ell_{1}$-ball $\paramspace = \big\{ \weights' \in \reals^{\dimlocalmodel} : \normgeneric{\weights'}{1} \leq \tau \big\}$. Since the $\ell_{1}$-ball is not a subspace, this projection is not a linear map (see Fig. 1).

Figure 1 of the entry projection — Figure 1: Projection of a vector $\weights \in \reals^{2}$ onto the $\ell_{1}$-ball $\paramspace = \big\{ \weights' \in \reals^{2} : \normgeneric{\weights'}{1} \leq \tau \big\}$. The projection $\projection{\paramspace}{\weights}$ is the point of $\paramspace$ closest to $\weights$ in the Euclidean norm

See also: Euclidean space, vector, minimum, convex, orthogonal projection, projected GD.

References

Boyd and Vandenberghe (2004). Convex Optimization. Cambridge Univ. Press.
Tibshirani (1996). Regression Shrinkage and Selection via the Lasso. J. Roy. Statist. Soc.: Ser. B (Methodological).

Cite this entry

@misc{dictml_projection,
  author = {Jung, Alexander},
  title = {projection},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-08-06},
  url = {https://dictionaryofml.org/terms/projection.html}
}