# robust principal component analysis (robust PCA)

*Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-11-08. [HTML](https://dictionaryofml.org/terms/rpca.html) · [typeset PDF](https://dictionaryofml.org/terms/rpca.pdf) · [LaTeX](https://dictionaryofml.org/terms/rpca.tex) · [Python demo](https://dictionaryofml.org/terms/rpca.py) · [notebook](https://dictionaryofml.org/terms/rpca.ipynb)*

Math is LaTeX with the book's macros expanded, so no notation package is needed to read it.

## Abstract

Robust principal component analysis (robust PCA) decomposes a feature matrix into a low-rank matrix, whose rows lie in a subspace of small dimension, and a sparse matrix that carries a few corrupted feature values. Unlike principal component analysis, it does not compute the low-rank matrix from the recorded values, so a large fault lands in the sparse matrix instead of turning the subspace. One approach states robust PCA as a convex optimization problem, referred to as principal component pursuit. Convex optimization methods solve it, among them the alternating direction method of multipliers. The resulting method delivers estimates for the principal directions that span the subspace, and for the corrupted entries in the features of each data point.

## Definition

The weather station Krems an der Donau records the air temperature every hour, so ten days of 2024 give ten rows of $24$ values (Fig. 1). Stacking the $24$ values of one day gives the feature vector ${\mathbf x} \in \mathbb{R}^{d}$, with $d = 24$, of the corresponding data point. Stacking the $m = 10$ feature vectors row-wise gives the $m \times d$ feature matrix ${\mathbf X}$. Each feature vector is centered by subtracting the average of the $m$ feature vectors, so that this average is the zero vector.

Principal component analysis uses the feature matrix to learn a given number $d'$ of orthonormal vectors $\mathbf{u}^{(1)}, \ldots, \mathbf{u}^{(d')}$, the principal directions of ${\mathbf X}$. They span a $d'$-dimensional subspace $\mathcal{U}$ of $\mathbb{R}^{d}$ such that each feature vector ${\mathbf x}^{(r)}$, for $r = 1, \ldots, m$, can be well approximated by its projection

$$
\mathbf{P} {\mathbf x}^{(r)} =
\sum_{j=1}^{d'}
\langle \mathbf{u}^{(j)}, {\mathbf x}^{(r)} \rangle
\mathbf{u}^{(j)} \text{.}
$$

Here, $\mathbf{P} \in \mathbb{R}^{d \times d}$ denotes the orthogonal projection matrix on the subspace $\mathcal{U}$. It is given explicitly by $\mathbf{P}=\sum_{j=1}^{d'}
\mathbf{u}^{(j)} \big( \mathbf{u}^{(j)} \big)^{\top}$ and provides a fully equivalent representation of the subspace $\mathcal{U}$ spanned by the principal directions.

The projection matrix $\mathbf{P}$ delivered by principal component analysis decomposes each feature vector as ${\mathbf x}^{(r)} = \mathbf{P} {\mathbf x}^{(r)} + {\mathbf e}^{(r)}$, with the approximation error ${\mathbf e}^{(r)}$. These decompositions (one for each $r=1,\ldots,m$) can be expressed more compactly in terms of the feature matrix as

\begin{equation}
\tag{1}
{\mathbf X} = {\mathbf L} + \mathbf{E} \text{, where } {\mathbf L} = {\mathbf X} \mathbf{P}
\text{ and } \mathbf{E} = {\mathbf X} - {\mathbf L} \text{.}
\end{equation}

The rank of ${\mathbf L}$ is at most $d'$, since every row of ${\mathbf L}$ lies in the subspace $\mathcal{U}$. The error matrix $\mathbf{E}$ carries the ${\mathbf e}^{(r)}$ as its rows, so $\left\lVert \mathbf{E} \right\rVert_{F}^{2} = \sum_{r=1}^{m}
\left\lVert {\mathbf e}^{(r)} \right\rVert_{2}^{2}$. Principal component analysis chooses $\mathbf{P}$ to make this quantity as small as possible. In this reading, principal component analysis splits the feature matrix into a low-rank part ${\mathbf L}$ and a small remainder $\mathbf{E}$.

The decomposition Eq. (1) suggests using a representation of the component $\mathbf{P} {\mathbf x}^{(r)}$ as new features. One such representation is the vector ${\mathbf z}^{(r)} = \big(
\langle \mathbf{u}^{(1)}, {\mathbf x}^{(r)} \rangle, \ldots,
\langle \mathbf{u}^{(d')}, {\mathbf x}^{(r)} \rangle
\big)^{\top}$ whose entries are the inner products between ${\mathbf x}^{(r)}$ and the principal directions of the feature matrix ${\mathbf X}$. For $d' = 2$ the new features $z_{1}, z_{2}$ can be used as coordinates in a scatterplot, as panel (a) of Fig. 2 shows.

Sometimes a feature value is faulty, for example because a temperature sensor fails. The red square in Fig. 1 marks such a failure: the noon reading of day five is increased by $22$ degrees, which gives the perturbed feature vector $\widetilde{{\mathbf x}}^{(5)}$ and, in turn, the perturbed feature matrix $\widetilde{{\mathbf X}}$. Panel (b) of Fig. 2 shows the scatterplot that principal component analysis delivers from $\widetilde{{\mathbf X}}$. The new features $z^{(5)}_{1}, z^{(5)}_{2}$ obtained for $\widetilde{{\mathbf x}}^{(5)}$ are far from those of the other days, and the second learned principal direction turns toward the axis of the perturbed feature. Thus, an isolated but large fault can substantially distort the coordinates of every data point.

The decomposition Eq. (1) explains why. Applied to $\widetilde{{\mathbf X}}$, it has the low-rank part ${\mathbf L} = \widetilde{{\mathbf X}} \mathbf{P}$, which is computed from the recorded values, and a remainder $\mathbf{E}$ that principal component analysis keeps small in the Frobenius norm. The $22$ degrees can therefore not be left in $\mathbf{E}$. Most of them have to appear in ${\mathbf L}$, and the only way a matrix of rank at most $d'$ can carry a large value at a single entry is for the subspace $\mathcal{U}$ to turn toward the faulty feature.

![Figure 1 of the entry rpca](https://dictionaryofml.org/terms/fig/rpca-fig1.svg)

*Figure 1: Hourly temperature recordings for ten days of 2024 at Krems an der Donau, drawn as small black dots. Each day is a data point with a $24$-dimensional feature vector ${\mathbf x}$. The red circle marks the correct noon reading of day five, and the large red square the perturbation that replaces it, $22$ degrees higher*

Robust principal component analysis drops the constraint ${\mathbf L} = {\mathbf X} \mathbf{P}$ and adds a third summand for the faulty entries,

\begin{equation}
\tag{2}
{\mathbf X} = {\mathbf L} + \mathbf{S} + \mathbf{E} \text{.}
\end{equation}

The low-rank matrix ${\mathbf L}$ is now an unknown in its own right, constrained only by $\mathop{\mathrm{rank}}({\mathbf L}) \leq d'$. The sparse matrix $\mathbf{S}$ carries the faulty feature values and is zero in every other entry. The remainder $\mathbf{E}$ stays small, as in the principal component analysis decomposition Eq. (1). Row $r$ of the decomposition Eq. (2) splits the feature vector ${\mathbf x}^{(r)}$ into these three parts.

The subspace and the principal directions are computed from ${\mathbf L}$ afterwards. The rows of ${\mathbf L}$ span a subspace of dimension at most $d'$, and the principal directions $\mathbf{u}^{(1)}, \ldots, \mathbf{u}^{(d')}$ are the leading right singular vectors of ${\mathbf L}$, delivered by its singular value decomposition. The projection matrix $\mathbf{P}$ follows from them, and ${\mathbf L} = {\mathbf L} \mathbf{P}$ holds because every row of ${\mathbf L}$ already lies in the subspace that $\mathbf{P}$ projects on. Robust principal component analysis therefore delivers the same objects as principal component analysis, but from a matrix ${\mathbf L}$ that is free of the faulty feature values, since $\mathbf{S}$ carries them.

The goal of making $\mathbf{E}$ small while $\mathbf{S}$ carries only a few non-zero entries can be stated as an optimization problem

\begin{equation}
\tag{3}
\min_{{\mathbf L}, \mathbf{S} \in \mathbb{R}^{m \times d}}
\left\lVert {\mathbf X} - {\mathbf L} - \mathbf{S} \right\rVert_{F} \quad
\text{ subject to } \mathop{\mathrm{rank}}({\mathbf L}) \leq d', \;
\left\lVert \mathbf{S} \right\rVert_{0} \leq s \text{.}
\end{equation}

The optimization variables are the two matrices ${\mathbf L}$ and $\mathbf{S}$, and the objective function is the Frobenius norm of the remainder $\mathbf{E} = {\mathbf X} - {\mathbf L} - \mathbf{S}$. The pseudo-norm $\left\lVert \mathbf{S} \right\rVert_{0}$ denotes the number of non-zero entries of $\mathbf{S}$, and $s$ is a given bound on the number of faulty feature values.

The optimization problem Eq. (3) is non-convex, since neither the rank nor the number of non-zero entries is a convex function. Principal component pursuit replaces it with the convex optimization problem

\begin{equation}
\tag{4}
\min_{{\mathbf L}, \mathbf{S} \in \mathbb{R}^{m \times d}}
\left\lVert {\mathbf L} \right\rVert_{*} + \alpha \left\lVert \mathbf{S} \right\rVert_{1} \quad
\text{ subject to } {\mathbf X} = {\mathbf L} + \mathbf{S} \text{,}
\end{equation}

with a regularization parameter $\alpha > 0$ (Candès et al., 2011). Here, $\left\lVert {\mathbf L} \right\rVert_{*}$ is the sum of the singular values of ${\mathbf L}$, and $\left\lVert \mathbf{S} \right\rVert_{1}$ the sum of the absolute values of the entries of $\mathbf{S}$. These two norms are convex, as every norm is (Boyd and Vandenberghe, 2004, Sects. 3.1.5 and A.1.6), and take the place of the rank of ${\mathbf L}$ and of the number of non-zero entries of $\mathbf{S}$. A large regularization parameter $\alpha$ makes every non-zero entry of $\mathbf{S}$ expensive, so the solution keeps $\mathbf{S}$ sparser and leaves more of ${\mathbf X}$ to ${\mathbf L}$. The rank of ${\mathbf L}$, and with it the dimension $d'$ of the subspace, then comes out larger, while a small $\alpha$ has the opposite effect.

Robust principal component analysis methods differ in the optimization method used to solve principal component pursuit Eq. (4). Algorithm 1 summarizes one such method, obtained by applying the alternating direction method of multipliers. Panel (c) of Fig. 2 shows the scatterplot that principal component analysis delivers from the low-rank matrix ${\mathbf L}$ that Algorithm 1 recovers from the perturbed feature matrix $\widetilde{{\mathbf X}}$. The ten days are back at the coordinates of panel (a). The non-zero entries of $\mathbf{S}$ point at the faulty readings themselves, which is why robust principal component analysis is also used for anomaly detection.

![Figure 2 of the entry rpca](https://dictionaryofml.org/terms/fig/rpca-fig2.svg)

*Figure 2: Coordinates of the same ten days given by the two features learned by (a) principal component analysis on the clean readings, (b) principal component analysis after one noon reading of day five is increased by $22$ degrees, and (c) principal component analysis on the low-rank matrix ${\mathbf L}$ that robust principal component analysis recovers from the corrupted readings. A cross marks day five in every panel, a filled circle each of the other nine days. All three panels share one set of axis limits, so the point clouds can be compared directly*

Algorithm 1 is a fixed-point iteration

$$
\big({\mathbf L}^{(t+1)}, \mathbf{S}^{(t+1)},
{\mathbf W}^{(t+1)}\big) = T\big({\mathbf L}^{(t)}, \mathbf{S}^{(t)},
{\mathbf W}^{(t)}\big) \text{,}
$$

whose operator $T$ chains two proximal operators with the update of the matrix ${\mathbf W}$. Shrinking the singular values of ${\mathbf B}$ by $1/\rho$ is the proximal operator of the nuclear norm (Candès et al., 2011, Eq. (5.3)), which is why that step needs the singular value decomposition. Shrinking every entry of ${\mathbf C}$ by $\alpha/\rho$ is the proximal operator of $\left\lVert \cdot \right\rVert_{1}$ (Candès et al., 2011, Eq. (5.2)), the one that solvers for the least absolute shrinkage and selection operator also use and the one step where $\alpha$ enters the iteration. At a fixed point of $T$ the three updates leave the iterate unchanged. This happens precisely when ${\mathbf X} = {\mathbf L} + \mathbf{S}$ holds, $({\mathbf L}, \mathbf{S})$ solves principal component pursuit Eq. (4), and $\rho {\mathbf W}$ is a Lagrange multiplier of the constraint. The optimization problem Eq. (4) is convex, so the iteration converges for every $\rho > 0$: the residual ${\mathbf X} - {\mathbf L} - \mathbf{S}$ tends to zero and the objective function to its minimum (Boyd et al., 2011, Sect. 3.2). For the averages of the iterates over the first $T$ iterations, this convergence holds with an explicit bound (Beck, 2017, Thm. 15.4).

![Algorithm 1 of the entry rpca](https://dictionaryofml.org/terms/fig/rpca-alg1.svg)

*Algorithm 1: Principal component pursuit by alternating direction method of multipliers (Candès et al., 2011, Sect. 5, Algorithm 1)*

**See also:** [principal component analysis](https://dictionaryofml.org/terms/pca.html), sparse principal component analysis, probabilistic principal component analysis, [singular value decomposition](https://dictionaryofml.org/terms/svd.html), rank, alternating direction method of multipliers, proximal operator, fixed-point iteration, least absolute shrinkage and selection operator, [regularization](https://dictionaryofml.org/terms/regularization.html), outlier, anomaly detection, robustness.

## References

- Candès et al. (2011). Robust Principal Component Analysis?. *Journal of the ACM*. <https://doi.org/10.1145/1970392.1970395>
- Boyd and Vandenberghe (2004). Convex Optimization. *Cambridge Univ. Press*. <https://doi.org/10.1017/CBO9780511804441>
- Boyd et al. (2011). Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. *Found. Trends Mach. Learn.*. <https://doi.org/10.1561/2200000016>
- Beck (2017). First-Order Methods in Optimization. *SIAM-Society for Industrial and Applied Mathematics*.

## Cite this entry

```bibtex
@misc{dictml_rpca,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {robust principal component analysis (robust PCA)},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-11},
  url = {https://dictionaryofml.org/terms/rpca.html}
}
```