Dictionary of Applied Machine Learning

eigenvalue

Updated on 2026-09-19

Typeset PDF version — the authoritative form of this entry

Python demo — a script that recomputes what this entry states and prints one line per check

An eigenvalue of a square matrix $\mA$ is a number $\eigvalgen$ for which some nonzero vector $\vx$ satisfies $\mA\vx = \eigvalgen\vx$. Such a vector is an eigenvector of $\mA$. Multiplying an eigenvector by the matrix rescales it without moving it off the line it spans. The eigenvalues of a diagonalizable matrix form the diagonal factor of its eigenvalue decomposition (EVD). An iterative method that applies an affine update, such as gradient descent (GD) for linear regression, converges to the unique fixed point of the update, for every initialization, exactly when each eigenvalue of the update matrix has magnitude less than one. The second-smallest eigenvalue of the Laplacian matrix of an undirected graph measures how well the graph is connected and underlies spectral clustering.

Definition

P-convA square matrix $\mA$ represents a linear map from $\reals^{\featuredim}$ to $\reals^{\featuredim}$; the image of a vector $\vx$ under this map is the product $\mA\vx$. Multiplying a vector by a square matrix normally moves it off the line it spans. Some nonzero vectors are exceptions: each stays on the line it spans and is only rescaled. The factor by which such a vector is rescaled is an eigenvalue. Formally, a number $\eigvalgen \in \reals$ is an eigenvalue of a square matrix $\mA \in \reals^{\featuredim \times \featuredim}$, one with $\featuredim$ rows and $\featuredim$ columns, if $\mA \vx = \eigvalgen \vx$ for some nonzero vector $\vx \in \reals^{\featuredim}$; such a vector is an eigenvector of $\mA$. The product $\mA\vx$ is the image of $\vx$ under $\mA$; for an eigenvector it stays on the line $\vx$ spans, pointing the same way when $\eigvalgen > 0$ and the opposite way when $\eigvalgen < 0$ (see Fig. 1).

Figure 1 of the entry eigenvalue
Figure 1: An eigenvector $\vx$ of $\mA$ (solid arrow) and its image $\mA \vx = \eigvalgen \vx$ (dashed arrow), here with eigenvalue $\eigvalgen = 1.8$. The image stays on the dotted line that $\vx$ spans, rescaled by the eigenvalue $\eigvalgen$. A negative eigenvalue $\eigvalgen$ would point the image the other way along the same line
Eigenvalues can be used to study iterative machine learning (ML) methods with an update of the form \[ \weights^{(\iteridx+1)} = \mM \weights^{(\iteridx)} + \vb \text{.} \] An example is gradient descent (GD) for linear regression, whose update operator $\gdstep{\lrate} : \weights \mapsto \mM \weights + \vb$ is affine with iteration matrix $\mM = \mI - (2\lrate/\samplesize) \featuremtx^{\top}\featuremtx$, where $\mI$ is the identity matrix, $\lrate$ is the learning rate, $\samplesize$ the number of data points and $\featuremtx$ the feature matrix of the training set (see linear regression). Such an iteration converges to the unique fixed point $\widehat{\weights}$ of $\gdstep{\lrate}$, for every initialization, if and only if the spectral radius $\rho$ of $\mM$, the largest magnitude of an eigenvalue of $\mM$, satisfies $\rho < 1$ (Golub and Loan, 2013).

P-graphBeyond the convergence of iterative methods, eigenvalues are also used to measure the connectivity of an undirected graph $\graph$ with $\nrnodes$ nodes. The Laplacian matrix $\LapMat{\graph}$ of $\graph$ is positive semi-definite (psd), with eigenvalues $0 = \eigval{1} \leq \eigval{2} \leq \ldots \leq \eigval{\nrnodes}$. The second-smallest eigenvalue $\eigval{2}$, the algebraic connectivity of $\graph$, quantifies connectivity: $\eigval{2} > 0$ if and only if $\graph$ is connected, and $\eigval{2}$ close to zero indicates two clusters of nodes with few edges between them (Chung, 1997; Fiedler, 1973; Luxburg, 2007). Fig. 2 shows three edge sets on the same six nodes: adding edges increases $\eigval{2}$ from $0$, for two clusters without a connecting edge, to $6$, for the complete graph. The eigenvector corresponding to $\eigval{2}$, referred to as the Fiedler vector, can be used to partition $\graph$ into two clusters: each node is assigned according to the sign of its own entry in the Fiedler vector (Fiedler, 1975; Spielman, 2025).

Figure 2 of the entry eigenvalue
Figure 2: Three edge sets on the same six nodes, with increasing connectivity from left to right, and the second-smallest eigenvalue $\eigval{2}$ of the corresponding Laplacian matrix. (a) two clusters without a connecting edge, so the graph is disconnected and $\eigval{2} = 0$. (b) one edge connects the clusters, so the graph is connected and $\eigval{2} \approx 0.44$ is positive but close to zero. (c) the complete graph, where every pair of nodes is connected, so the graph is connected and $\eigval{2} = 6$ attains the largest possible value. Data generated by pythondemos/eigenvalue.py
See also: matrix, eigenvector, eigenvalue decomposition (EVD), spectral radius, GD, linear regression, contractive operator, fixed-point iteration, Laplacian matrix, algebraic connectivity, Fiedler vector, spectral clustering.

References

  1. Golub and Loan (2013). Matrix Computations. The Johns Hopkins Univ. Press.
  2. Chung (1997). Spectral Graph Theory. American Mathematical Society.
  3. Fiedler (1973). Algebraic connectivity of graphs. Czechoslovak Mathematical Journal.
  4. Luxburg (2007). A tutorial on spectral clustering. Statist. Comput.. doi.org/10.1007/s11222-007-9033-z
  5. Fiedler (1975). A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory. Czechoslovak Mathematical Journal.
  6. Spielman (2025). Spectral and Algebraic Graph Theory (Incomplete Draft). Ebook. cs-www.cs.yale.edu/homes/spielman/sagt

Cite this entry

@misc{dictml_eigenvalue,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {eigenvalue},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-09-21},
  url = {https://dictionaryofml.org/terms/eigenvalue.html}
}