Dictionary of Applied Machine Learning

Gaussian mixture model

Updated on 2026-09-06

Typeset PDF version — the authoritative form of this entry

A Gaussian mixture model (GMM) is a probabilistic model for data points with numeric feature vectors. Each data point is generated by first drawing a latent cluster index according to cluster probabilities and then drawing the feature vector from the multivariate normal distribution of that cluster; the marginal distribution is a weighted sum of multivariate normal distributions. The model parameters — cluster probabilities, means, and covariance matrices — are learned by maximum likelihood, typically via the expectation–maximization (EM) algorithm. A fitted GMM delivers soft clustering: the posterior distribution of the latent index grades the membership of each data point in every cluster.

Definition

The nightly minimum temperatures recorded over one year at a weather station do not scatter around a single typical value: cold-season and warm-season nights form two distinct regimes. A GMM is a probabilistic model that captures such multi-regime data: it models the generation of data points with numeric feature vectors $\featurevec \in \reals^{\nrfeatures}$ (Bishop, 2006; Murphy, 2012). It assumes that each data point is generated by first drawing a latent cluster index $I \in \{1,\,\ldots,\,\nrcluster\}$ according to cluster probabilities \[\prob{I=\clusteridx} = p_{\clusteridx}, \qquad \sum_{\clusteridx=1}^{\nrcluster} p_{\clusteridx}=1\text{.}\] Conditioned on $I=\clusteridx$, the feature vector $\featurevec$ is drawn from a multivariate normal distribution $\probdist^{(\clusteridx)}= \mvnormal{\meanvec{\clusteridx}}{\covmtx{\clusteridx}}$. The resulting marginal distribution of $\featurevec$ is therefore a weighted sum of multivariate normal distributions (Fig. 1), i.e., \[\probdist =\sum_{\clusteridx=1}^{\nrcluster} p_{\clusteridx} \mvnormal{\meanvec{\clusteridx}}{\covmtx{\clusteridx}}\text{.}\]

Figure 1 of the entry gmm
Figure 1: Illustration of a GMM with three components

References

  1. Bishop (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media. doi.org/10.1007/978-0-387-45528-0
  2. Murphy (2012). Machine Learning: A Probabilistic Perspective. MIT Press.

Cite this entry

@misc{dictml_gmm,
  author = {Jung, Alexander and Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {Gaussian mixture model},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-09-11},
  url = {https://dictionaryofml.org/terms/gmm.html}
}