# unsupervised learning *Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-11-04. [HTML](https://dictionaryofml.org/terms/unsupervisedlearning.html) · [typeset PDF](https://dictionaryofml.org/terms/unsupervisedlearning.pdf) · [LaTeX](https://dictionaryofml.org/terms/unsupervisedlearning.tex) · [Python demo](https://dictionaryofml.org/terms/unsupervisedlearning.py) · [notebook](https://dictionaryofml.org/terms/unsupervisedlearning.ipynb)* Math is LaTeX with the book's macros expanded, so no notation package is needed to read it. ## Abstract Unsupervised learning refers to machine learning methods that do not require knowledge of the label of any data point. These methods learn a hypothesis by loss functions that are constructed from the features of data points only. Two important examples of unsupervised learning are clustering and dimensionality reduction. Clustering methods use some notion of clustering error to learn model parameters that encode a cluster structure. Dimensionality reduction methods construct a loss from the reconstruction error. ## Definition Twenty days at Krems an der Donau are each characterized by two features, the daily minimum and the daily maximum temperature, and nothing else is recorded about them (Fig. 1). Unsupervised learning methods learn a hypothesis map that predicts a label that is not known for any of these data points. For example, the label could be an index that selects one out of two clusters to which the data point belongs. Another label could be the optimal scalar representation of a data point. Formally, an unsupervised machine learning method learns a hypothesis out of a hypothesis space based on an unlabeled dataset $\mathcal{D} = \big\{ {\mathbf x}^{(1)}, \ldots, {\mathbf x}^{(m)} \big\}$ of feature vectors ${\mathbf x}^{(r)} \in \mathcal{X}$, with no label space. The model parameters are typically learned by an instance of empirical risk minimization, minimizing the average loss over the dataset, $$ \widehat{h} \in \mathop{\mathrm{arg\,min}}_{h \in \mathcal{H}} \frac{1}{m} \sum_{r=1}^{m} L\left({\mathbf x}^{(r)},h \right) \text{.} $$ What sets unsupervised methods apart from supervised methods, such as linear regression, is the choice of loss function. Two important families of unsupervised methods are obtained by specific constructions of the loss, clustering and dimensionality reduction. The clustering method k-means uses $k$ cluster centroids as model parameters ${\mathbf w} = \big\{ {{\mathbf w}}^{(1)}, \ldots, {{\mathbf w}}^{(k)} \big\}$ (Hastie et al., 2009, Sect. 14.3.6). They are learned by minimizing the average clustering error, the loss $$ L\left({\mathbf x},{\mathbf w} \right) = \min_{c = 1, \ldots, k} \left\lVert {\mathbf x} - {{\mathbf w}}^{(c)} \right\rVert_{2}^{2} \text{.} $$ Principal component analysis is a dimensionality reduction method that learns a linear feature transformation ${\mathbf x} \mapsto {\mathbf W} {\mathbf x}$ together with a reconstruction map ${\mathbf z} \mapsto {\mathbf R} {\mathbf z}$ that rebuilds the feature vector from it. Both are model parameters, and for a given ${\mathbf R}$ the loss of a data point is its reconstruction error $$ L\left({\mathbf x},\left( {\mathbf W},{\mathbf R} \right) \right) = \left\lVert {\mathbf x} - {\mathbf R} {\mathbf W} {\mathbf x} \right\rVert_{2}^{2} \text{.} $$ One ${\mathbf R}$ is used for the whole dataset, and empirical risk minimization learns it together with ${\mathbf W}$ by minimizing the average of this loss (Hastie et al., 2009, Sect. 14.5.1). For the learned pair the reconstruction is the orthogonal projection of the feature vector on a subspace of dimension $d'$. Fig. 1 shows both on the Krems days: the two cluster centroids with the data points assigned to each, and the learned direction with the orthogonal projections that give each day its single number. ![Figure 1 of the entry unsupervisedlearning](https://dictionaryofml.org/terms/fig/unsupervisedlearning-fig1.svg) *Figure 1: Twenty weather recordings at Krems an der Donau, each a data point with two features and no label. The clustering method k-means learned the two cluster centroids (filled squares), and the solid ellipses enclose the data points assigned to each. The dashed line carries the direction learned by a linear feature transformation. The filled triangles mark the orthogonal projections, which represent each data point by a single number, the location of the triangle along the dashed line. No label entered either computation. Data generated by pythondemos/unsupervisedlearning.py* **See also:** [clustering](https://dictionaryofml.org/terms/clustering.html), [$k$-means](https://dictionaryofml.org/terms/kmeans.html), cluster centroid, [dimensionality reduction](https://dictionaryofml.org/terms/dimred.html), [principal component analysis](https://dictionaryofml.org/terms/pca.html), feature transformation, [feature learning](https://dictionaryofml.org/terms/featlearn.html), [label](https://dictionaryofml.org/terms/label.html), [dataset](https://dictionaryofml.org/terms/dataset.html), [empirical risk minimization](https://dictionaryofml.org/terms/erm.html), [loss](https://dictionaryofml.org/terms/loss.html), [supervised learning](https://dictionaryofml.org/terms/supervisedlearning.html), [self-supervised learning](https://dictionaryofml.org/terms/selfsupervisedlearning.html), semi-supervised learning, clustering error. ## References - Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. *Springer Science+Business Media*. ## Cite this entry ```bibtex @misc{dictml_unsupervisedlearning, author = {Jung, Alexander}, editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard}, title = {unsupervised learning}, howpublished = {Dictionary of Applied Machine Learning (course edition)}, year = {2026}, doi = {10.5281/zenodo.21569296}, note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-04}, url = {https://dictionaryofml.org/terms/unsupervisedlearning.html} } ```