# eigenvalue

*Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-10-29. [HTML](https://dictionaryofml.org/terms/eigenvalue.html) · [typeset PDF](https://dictionaryofml.org/terms/eigenvalue.pdf) · [LaTeX](https://dictionaryofml.org/terms/eigenvalue.tex) · [Python demo](https://dictionaryofml.org/terms/eigenvalue.py) · [notebook](https://dictionaryofml.org/terms/eigenvalue.ipynb)*

Math is LaTeX with the book's macros expanded, so no notation package is needed to read it.

## Abstract

An eigenvalue of a square matrix ${\mathbf A}$ is a number $\lambda$ for which some nonzero vector ${\mathbf x}$ satisfies ${\mathbf A}{\mathbf x} = \lambda{\mathbf x}$. Such a vector is an eigenvector of ${\mathbf A}$. Multiplying an eigenvector by the matrix rescales it without moving it off the line it spans. The eigenvalues of a diagonalizable matrix form the diagonal factor of its eigenvalue decomposition. An iterative method that applies an affine update, such as gradient descent for linear regression, converges to the unique fixed point of the update, for every initialization, exactly when each eigenvalue of the update matrix has magnitude less than one. The second-smallest eigenvalue of the Laplacian matrix of an undirected graph measures how well the graph is connected and underlies spectral clustering.

## Definition

Whether gradient descent for linear regression converges is decided by the eigenvalues of a single square matrix. A square matrix ${\mathbf A}$ represents a linear map from $\mathbb{R}^{d}$ to $\mathbb{R}^{d}$; the image of a vector ${\mathbf x}$ under this map is the product ${\mathbf A}{\mathbf x}$. Multiplying a vector by a square matrix normally moves it off the line it spans. Some nonzero vectors are exceptions: each stays on the line it spans and is only rescaled. The factor by which such a vector is rescaled is an eigenvalue.

Formally, a number $\lambda \in \mathbb{R}$ is an eigenvalue of a square matrix ${\mathbf A} \in \mathbb{R}^{d \times d}$, one with $d$ rows and $d$ columns, if ${\mathbf A} {\mathbf x} = \lambda {\mathbf x}$ for some nonzero vector ${\mathbf x} \in \mathbb{R}^{d}$; such a vector is an eigenvector of ${\mathbf A}$ (Strang, 2016). For an eigenvector, the image ${\mathbf A}{\mathbf x}$ stays on the line ${\mathbf x}$ spans, pointing the same way when $\lambda > 0$ and the opposite way when $\lambda < 0$ (see Fig. 1). Arranging $d$ linearly independent eigenvectors of a diagonalizable matrix as the columns of $\mathbf{V}$, and the corresponding eigenvalues in a diagonal matrix ${\boldsymbol \Lambda}$, yields the eigenvalue decomposition ${\mathbf A} = \mathbf{V} {\boldsymbol \Lambda} \mathbf{V}^{-1}$.

![Figure 1 of the entry eigenvalue](https://dictionaryofml.org/terms/fig/eigenvalue-fig1.svg)

*Figure 1: An eigenvector ${\mathbf x}$ of ${\mathbf A}$ (solid arrow) and its image ${\mathbf A} {\mathbf x} = \lambda {\mathbf x}$ (dashed arrow), here with eigenvalue $\lambda = 1.8$. The image stays on the dotted line that ${\mathbf x}$ spans, rescaled by $\lambda$. A negative $\lambda$ would point the image the other way along the same line*

Eigenvalues can be used to study iterative machine learning methods with an update of the form

$$
{\mathbf w}^{(t+1)} = {\mathbf M} {\mathbf w}^{(t)} + {\mathbf b}
\text{.}
$$

An example is gradient descent for linear regression, whose update operator $\mathcal{F}^{(\eta)} : {\mathbf w} \mapsto {\mathbf M} {\mathbf w} + {\mathbf b}$ is affine with iteration matrix ${\mathbf M} = \mathbf{I} - (2\eta/m) {\mathbf X}^{\top}{\mathbf X}$. Here, $\mathbf{I}$ is the identity matrix, $\eta$ the learning rate, $m$ the number of data points, and ${\mathbf X}$ the feature matrix of the training set (see linear regression). Such an iteration converges to the unique fixed point $\widehat{{\mathbf w}}$ of $\mathcal{F}^{(\eta)}$, for every initialization, if and only if the spectral radius $\rho$ of ${\mathbf M}$, the largest magnitude of an eigenvalue of ${\mathbf M}$, satisfies $\rho < 1$ (Horn and Johnson, 2013, Thm. 5.6.12). For a diagonalizable ${\mathbf M}$, the eigenvalue decomposition shows why: in the basis of its eigenvectors, each iteration rescales the components of the error ${\mathbf w}^{(t)} - \widehat{{\mathbf w}}$ by the eigenvalues, so the error shrinks exactly when every eigenvalue has magnitude below one.

Beyond the convergence of iterative methods, eigenvalues are also used to measure the connectivity of an undirected graph $\mathcal{G}$ with the number of nodes $n$. The Laplacian matrix ${\mathbf L}^{(\mathcal{G})}$ of $\mathcal{G}$ is positive semi-definite, with eigenvalues $0 = \lambda_{1} \leq \lambda_{2} \leq \ldots \leq \lambda_{n}$. The second-smallest eigenvalue $\lambda_{2}$, the algebraic connectivity of $\mathcal{G}$, quantifies connectivity: $\lambda_{2} > 0$ if and only if $\mathcal{G}$ is connected, and $\lambda_{2}$ close to zero indicates two clusters of nodes with few edges between them (Chung, 1997; Fiedler, 1973; Luxburg, 2007). Fig. 2 shows three edge sets on the same six nodes: adding edges increases $\lambda_{2}$ from $0$, for two clusters without a connecting edge, to $6$, for the complete graph.

![Figure 2 of the entry eigenvalue](https://dictionaryofml.org/terms/fig/eigenvalue-fig2.svg)

*Figure 2: Three edge sets on the same six nodes, with increasing connectivity from left to right, and the second-smallest eigenvalue $\lambda_{2}$ of the corresponding Laplacian matrix. (a) Two clusters without a connecting edge, so the graph is disconnected and $\lambda_{2} = 0$. (b) One edge connects the clusters, so the graph is connected and $\lambda_{2} \approx 0.44$ is positive but close to zero. (c) The complete graph, where every pair of nodes is connected, so the graph is connected and $\lambda_{2} = 6$ attains the largest possible value, the number of nodes $n = 6$, since no eigenvalue of the Laplacian matrix of a graph with $n$ nodes exceeds $n$. Data generated by pythondemos/eigenvalue.py*

The eigenvector corresponding to $\lambda_{2}$, referred to as the Fiedler vector, can be used to partition $\mathcal{G}$ into two clusters: each node is assigned according to the sign of its own component of the Fiedler vector (Fiedler, 1975; Spielman, 2025).

**See also:** [matrix](https://dictionaryofml.org/terms/matrix.html), eigenvector, [eigenvalue decomposition](https://dictionaryofml.org/terms/evd.html), spectral radius, [gradient descent](https://dictionaryofml.org/terms/gd.html), [linear regression](https://dictionaryofml.org/terms/linreg.html), contractive operator, fixed-point iteration, Laplacian matrix, algebraic connectivity, Fiedler vector, spectral clustering.

## References

- Strang (2016). Introduction to Linear Algebra. *Wellesley-Cambridge Press*.
- Horn and Johnson (2013). Matrix Analysis. *Cambridge Univ. Press*. <https://doi.org/10.1017/cbo9781139020411>
- Chung (1997). Spectral Graph Theory. *American Mathematical Society*.
- Fiedler (1973). Algebraic connectivity of graphs. *Czechoslovak Mathematical Journal*.
- Luxburg (2007). A tutorial on spectral clustering. *Statist. Comput.*. <https://doi.org/10.1007/s11222-007-9033-z>
- Fiedler (1975). A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory. *Czechoslovak Mathematical Journal*.
- Spielman (2025). Spectral and Algebraic Graph Theory (Incomplete Draft). *Ebook*. <http://cs-www.cs.yale.edu/homes/spielman/sagt>

## Cite this entry

```bibtex
@misc{dictml_eigenvalue,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {eigenvalue},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-04},
  url = {https://dictionaryofml.org/terms/eigenvalue.html}
}
```