matrix

Typeset PDF version — the authoritative form of this entry

A matrix is a rectangular array of numbers arranged in rows and columns. The entry $A_{\sampleidx,\featureidx}$ of a matrix $\mA \in \reals^{\samplesize \times \nrfeatures}$ sits in row $\sampleidx$ and column $\featureidx$. A prominent example in machine learning (ML) is the feature matrix of a dataset, obtained by stacking the feature vectors of $\samplesize$ data points row-wise. A matrix represents several distinct mathematical objects: a system of linear equations, such as the normal equations of linear regression; a linear map between two vector spaces, after fixing a basis for each; or a tabular dataset with one row per data point and one column per feature. Matrices are the two-dimensional special case of an array.

Definition

Consider a tabular dataset of $\samplesize$ data points, each described by $\nrfeatures$ numerical features. Stacking the feature vectors row-wise produces an $\samplesize \times \nrfeatures$ rectangular array of numbers. This two-dimensional numeric array is referred to as the feature matrix $\featuremtx$ of the dataset.

More generally, a matrix of size $\samplesize \times \nrfeatures$ is a two-dimensional array of numbers denoted by \[ \mA = \begin{pmatrix} A_{1,1} & A_{1,2} & \dots & A_{1,\nrfeatures} \\ A_{2,1} & A_{2,2} & \dots & A_{2,\nrfeatures} \\ \vdots & \vdots & \ddots & \vdots \\ A_{\samplesize,1} & A_{\samplesize,2} & \dots & A_{\samplesize,\nrfeatures} \end{pmatrix} \in \mathbb{R}^{\samplesize \times \nrfeatures}. \] Here, $A_{\sampleidx,\featureidx}$ denotes the matrix entry in the $\sampleidx$-th row and the $\featureidx$-th column. Matrices are the two-dimensional special case of the more general array.

Matrices represent several distinct mathematical objects (Strang, 2016), including the following:

Systems of linear equations: A matrix can represent a system of linear equations \[ \begin{pmatrix} A_{1,1} & A_{1,2} \\ A_{2,1} & A_{2,2} \end{pmatrix} \begin{pmatrix} w_1 \\ w_2 \end{pmatrix} =\begin{pmatrix} y_1 \\ y_2 \end{pmatrix} \text{ compactly as } \mA \vw = \vy. \] One important example is the normal equations $\featuremtx^{\top}\featuremtx\,\weights = \featuremtx^{\top}\labelvec$ that give the optimal model parameters in linear regression.
Linear maps: Consider two vector spaces $\mathcal{U}$ and $\vecspace$, of dimension $\nrfeatures$ and $\samplesize$, respectively. Fixing a basis $\mathbf{u}^{(1)},\,\ldots,\,\mathbf{u}^{(\nrfeatures)}$ for $\mathcal{U}$ and a basis $\mathbf{v}^{(1)},\,\ldots,\,\mathbf{v}^{(\samplesize)}$ for $\vecspace$, each matrix $\mA \in \mathbb{R}^{\samplesize \times \nrfeatures}$ naturally defines a linear map $\alpha: \mathcal{U} \rightarrow \vecspace$ (see Fig.\ 1) such that \[\vu^{(\featureidx)} \mapsto \sum_{\sampleidx=1}^{\samplesize} A_{\sampleidx,\featureidx} \vv^{(\sampleidx)}.\]
Datasets: as in the opening example, a matrix can represent a dataset: each row a single data point, each column a specific feature or label.

Figure 1 of the entry matrix — Figure 1: Matrix $\mA$ defining a linear map between two vector spaces

See also: linear map, dataset, linear model, vector, feature matrix, normal equations, convolutional neural network (CNN).

References

Strang (2016). Introduction to Linear Algebra. Wellesley-Cambridge Press.

Cite this entry

@misc{dictml_matrix,
  author = {Jung, Alexander},
  title = {matrix},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-08-06},
  url = {https://dictionaryofml.org/terms/matrix.html}
}