# matrix

*Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-10-29. [HTML](https://dictionaryofml.org/terms/matrix.html) · [typeset PDF](https://dictionaryofml.org/terms/matrix.pdf) · [LaTeX](https://dictionaryofml.org/terms/matrix.tex) · [Python demo](https://dictionaryofml.org/terms/matrix.py) · [notebook](https://dictionaryofml.org/terms/matrix.ipynb)*

Math is LaTeX with the book's macros expanded, so no notation package is needed to read it.

## Abstract

A matrix is a two-dimensional array of numbers, arranged in rows and columns. The entry $A_{r,j}$ of a matrix ${\mathbf A} \in \mathbb{R}^{m \times d}$ sits in row $r$ and column $j$. A prominent example in machine learning is the feature matrix of a dataset, obtained by stacking the feature vectors of $m$ data points row-wise. A matrix represents several distinct mathematical objects. One is a system of linear equations, such as the normal equations of linear regression. Another is a linear map between two vector spaces, after fixing a basis for each. A third is a tabular dataset with one row per data point and one column per feature.

## Definition

Consider a spreadsheet of daily weather records at one station: each row holds a single day, each column one measurement such as temperature or humidity. Such a tabular dataset holds $m$ data points, each described by the same number $d$ of numerical features. The features of the $r$-th data point are collected in its feature vector ${\mathbf x}^{(r)} = \big(x^{(r)}_{1},
\ldots, x^{(r)}_{d}\big)^{\top} \in
\mathbb{R}^{d}$, one entry per column of the spreadsheet. Stacking these $m$ feature vectors row-wise, so that the $r$-th row is $\big({\mathbf x}^{(r)}\big)^{\top}$, produces an $m \times d$ rectangular array of numbers. Its entry in row $r$ and column $j$ is the $j$-th feature of the $r$-th data point. This two-dimensional numeric array is referred to as the feature matrix ${\mathbf X}$ of the dataset; Fig. 1 carries out the construction on three days of the weather example.

![Figure 1 of the entry matrix](https://dictionaryofml.org/terms/fig/matrix-fig1.svg)

*Figure 1: A tabular dataset of $m = 3$ data points and the feature matrix built from it. Each row of the table is one day, whose $d = 3$ measurements, temperature (temp.), humidity (humid.), and pressure (press.), form the feature vector ${\mathbf x}^{(r)} \in \mathbb{R}^{3}$; the day column indexes the data points and is not itself a feature, so it does not appear in ${\mathbf X}$. Stacking ${\mathbf x}^{(1)}, {\mathbf x}^{(2)}, {\mathbf x}^{(3)}$ as rows makes the entry $X_{r,j}$ the $j$-th measurement taken on day $r$: $X_{2,3} = 1008$ is the pressure on day $2$*

More generally, a matrix of size $m \times d$ is a two-dimensional array of numbers denoted by

$$
{\mathbf A} = \begin{pmatrix}
A_{1,1} & A_{1,2} & \dots & A_{1,d} \\
A_{2,1} & A_{2,2} & \dots & A_{2,d} \\
\vdots & \vdots & \ddots & \vdots \\
A_{m,1} & A_{m,2} & \dots & A_{m,d}
\end{pmatrix} \in \mathbb{R}^{m \times d}\text{.}
$$

Here, $A_{r,j}$ denotes the matrix entry in the $r$-th row and the $j$-th column. A digital image is a matrix of exactly this kind. Recording how green each pixel is turns a $16 \times 16$ image into a $16 \times 16$ matrix $\mathbf{G}$ whose two indices are pixel positions (Fig. 2).

![Figure 2 of the entry matrix](https://dictionaryofml.org/terms/fig/matrix-fig2.svg)

*Figure 2: A $16 \times 16$ pixel image of the ``for all'' quantifier $\forall$ and the matrix of its green channel. Pixel $(r,j)$ has greenness $G_{r, j} = 1$ where the glyph is drawn and $0$ elsewhere, so the $1$s trace the same shape in $\mathbf{G}$ that the shaded squares trace in the image*

Matrices represent several distinct mathematical objects (Strang, 2016), including the following:

- Systems of linear equations: A matrix collects the coefficients of a system of linear equations,

$$
\begin{pmatrix}
A_{1,1} & A_{1,2} \\
A_{2,1} & A_{2,2}
\end{pmatrix}
\begin{pmatrix}
x_1 \\
x_2
\end{pmatrix}
=\begin{pmatrix}
b_1 \\
b_2
\end{pmatrix}
\text{, written compactly as } {\mathbf A} {\mathbf x} = {\mathbf b} \text{.}
$$

Here, ${\mathbf x}$ is the vector of unknowns and ${\mathbf b}$ the right-hand side. One important example is the normal equations ${\mathbf X}^{\top}{\mathbf X}\,{\mathbf w} = {\mathbf X}^{\top}{\mathbf y}$, whose solutions are the model parameters that minimize the training error in linear regression, as the derivation following Fig. 3 shows.
- Linear maps: Consider two vector spaces $\mathcal{U}$ and $\mathcal{V}$, of dimension $d$ and $m$, respectively. Fixing a basis ${\mathbf u}^{(1)},\,\ldots,\,{\mathbf u}^{(d)}$ for $\mathcal{U}$ and a basis ${\mathbf v}^{(1)},\,\ldots,\,{\mathbf v}^{(m)}$ for $\mathcal{V}$, each matrix ${\mathbf A} \in \mathbb{R}^{m \times d}$ defines a linear map $f^{({\mathbf A})}: \mathcal{U} \rightarrow \mathcal{V}$ (see Fig. 3) via

$$
f^{({\mathbf A})}\big({\mathbf u}^{(j)}\big) = \sum_{r=1}^{m} A_{r,j} {\mathbf v}^{(r)}\text{.}
$$

Taking $\mathcal{U} = \mathbb{R}^{d}$ and $\mathcal{V} = \mathbb{R}^{m}$ with the standard bases, the image of a vector ${\mathbf x}$ under $f^{({\mathbf A})}$ is the matrix-vector product $f^{({\mathbf A})}({\mathbf x}) = {\mathbf A}{\mathbf x}$.
- Datasets: As in the opening example, a matrix can represent a dataset with each row holding a single data point and each column a specific feature or label.

![Figure 3 of the entry matrix](https://dictionaryofml.org/terms/fig/matrix-fig3.svg)

*Figure 3: The matrix ${\mathbf A}$ defines a linear map $f^{({\mathbf A})}$ between two vector spaces: the basis vector ${\mathbf u}^{(j)}$ is mapped to the vector (thick red arrows) whose coordinates form the $j$-th column of ${\mathbf A}$*

Written with the feature matrix, the training error that empirical risk minimization-based linear regression minimizes becomes a succinct algebraic expression. Collecting the labels of the training set in ${\mathbf y} \in \mathbb{R}^{m}$, the training error of the hypothesis $h^{({\mathbf w})}({\mathbf x}) := {\mathbf w}^{\top}{\mathbf x}$ is

$$
(1/m)\,\left\lVert {\mathbf X} {\mathbf w} - {\mathbf y} \right\rVert_{2}^{2}
\text{,}
$$

with gradient $(2/m)\,{\mathbf X}^{\top}({\mathbf X} {\mathbf w} - {\mathbf y})$. Each gradient descent step is therefore one multiplication by ${\mathbf X}$ followed by one by ${\mathbf X}^{\top}$, and equating the gradient to zero leads back to the normal equations from the first example above.

Matrices appear across machine learning beyond the feature matrix. The Jacobian matrix and the Hessian collect the first- and second-order partial derivatives of a multivariate function and are the input of gradient-based training. The covariance matrix records the pairwise covariances of a probability distribution over features. Principal component analysis, singular value decomposition, and a random projection factor or compress the feature matrix to lower its dimension. A kernel method replaces explicit feature vectors by a matrix of pairwise inner products, and a tabular Markov decision process stores its transition probabilities as a matrix. A graph is represented by matrices as well: its adjacency matrix lists which nodes are joined and with what edge weight. The Laplacian matrix assembled from those weights (Luxburg, 2007, Sect. 3) is the matrix whose eigenvectors graph clustering uses to split the nodes into clusters.

**See also:** linear map, [dataset](https://dictionaryofml.org/terms/dataset.html), linear model, vector, [feature matrix](https://dictionaryofml.org/terms/featuremtx.html), normal equations, array, transpose.

## References

- Strang (2016). Introduction to Linear Algebra. *Wellesley-Cambridge Press*.
- Luxburg (2007). A tutorial on spectral clustering. *Statist. Comput.*. <https://doi.org/10.1007/s11222-007-9033-z>

## Cite this entry

```bibtex
@misc{dictml_matrix,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {matrix},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-04},
  url = {https://dictionaryofml.org/terms/matrix.html}
}
```