# degree of belonging

*Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-11-19. [HTML](https://dictionaryofml.org/terms/dob.html) · [typeset PDF](https://dictionaryofml.org/terms/dob.pdf) · [LaTeX](https://dictionaryofml.org/terms/dob.tex) · [Python demo](https://dictionaryofml.org/terms/dob.py) · [notebook](https://dictionaryofml.org/terms/dob.ipynb)*

Math is LaTeX with the book's macros expanded, so no notation package is needed to read it.

## Abstract

A degree of belonging is a number that indicates the extent to which a data point belongs to a cluster, one for each pair of a data point and a cluster of a clustering. In the most common formulation, the degrees of belonging of one data point are nonnegative and sum to one, so they form a vector of the probability simplex and a probability mass function over the cluster indices. Hard clustering is the special case in which this vector has one entry equal to one and all others equal to zero, as delivered by k-means. A principled construction of degrees of belonging strictly between zero and one is the posterior distribution of the cluster index under a Gaussian mixture model, which the expectation–maximization algorithm computes in its E-step. The degrees of belonging in turn determine the clusters: the mean of a cluster is the average of the feature vectors weighted by them, and with degrees of belonging of zero or one this weighted average is the averaging step of Lloyd's algorithm. As the common variance of the clusters of a Gaussian mixture model shrinks to zero, the degrees of belonging turn into zero or one and soft clustering turns into hard clustering.

## Definition

Consider the $366$ days of one year at Krems an der Donau, each a data point with a feature vector ${\mathbf x} \in \mathbb{R}^{2}$ that holds the morning minimum and the maximum daytime temperature of the day. A clustering method decomposes these days into a cold cluster and a warm cluster. A day in January is a cold day and a day in July a warm day, but 28 May 2024, with a morning minimum of $11.1$ and a maximum of $21.3$ degrees, lies between the two. A degree of belonging $u_{c}^{(r)}$ is a number that indicates the extent to which data point $r$ belongs to cluster $c$, for $c = 1, \ldots, k$ and $r = 1,
\ldots, m$. For 28 May 2024, the Gaussian mixture model of Fig. 2 delivers the degrees of belonging $0.50$ to the cold and $0.50$ to the warm cluster, while a day in January obtains $1.00$ and $0.00$. In the most common formulation, the degrees of belonging of one data point are nonnegative and sum to one (Bezdek, 1981; Ruspini, 1969),

\begin{equation}
\tag{1}
u_{c}^{(r)} \geq 0
\quad \text{for } c = 1, \ldots, k
\qquad \text{and} \qquad
\sum_{c=1}^{k} u_{c}^{(r)}
= 1 \text{.}
\end{equation}

The vector ${\mathbf u}^{(r)} := \big(u_{1}^{(r)}, \ldots,
u_{k}^{(r)}\big)^{\top}$ then lies in the probability simplex $\Delta^{k-1}$. It is the probability mass function of a random variable with values in $\{1, \ldots, k\}$, the cluster index of the data point. Possibilistic clustering drops the sum constraint of the simplex condition Eq. (1) and lets each degree of belonging vary in $[0, 1]$ on its own (Krishnapuram and Keller, 1993). Fig. 1 sketches three data points and their degrees of belonging to two overlapping clusters.

![Figure 1 of the entry dob](https://dictionaryofml.org/terms/fig/dob-fig1.svg)

*Figure 1: Three data points with different degrees of belonging to two overlapping clusters, cluster $1$ (solid ellipse) and cluster $2$ (dashed ellipse). The feature vector ${\mathbf x}^{(1)}$ lies deep inside cluster $1$ and ${\mathbf x}^{(3)}$ deep inside cluster $2$, while ${\mathbf x}^{(2)}$ lies in the overlap with equal degrees of belonging. In every case $u_{1}^{(r)} + u_{2}^{(r)} = 1$, the simplex condition Eq. (1)*

Hard clustering is the special case in which every data point belongs to exactly one cluster. With $\widehat{c}^{(r)}$ the index of the cluster that data point $r$ is assigned to, its degrees of belonging are

\begin{equation}
\tag{2}
u_{c}^{(r)} =
\begin{cases}
1 & \text{for } c = \widehat{c}^{(r)}
\text{,} \\
0 & \text{otherwise} \text{,}
\end{cases}
\end{equation}

so the vector ${\mathbf u}^{(r)}$ is a vertex of the probability simplex $\Delta^{k-1}$ (Bishop, 2006, Sect. 9.1). k-means delivers such a hard clustering: $\widehat{c}^{(r)}$ is the index of the cluster centroid nearest to ${\mathbf x}^{(r)}$ in Euclidean distance. For the days at Krems and two cluster centroids, k-means assigns $192$ days to the cold cluster with cluster centroid $(1.5, 10.5)$ and $174$ days to the warm cluster with cluster centroid $(13.4, 26.1)$. The dashed curve in Fig. 2(b) shows these hard assignments over the year: they jump between $0$ and $1$ from one spring day to the next.

Soft clustering methods assign a data point to several clusters with degrees of belonging strictly between $0$ and $1$. A principled construction uses a probabilistic model for the data points, the Gaussian mixture model. Under it, the cluster index $I^{(r)} \in \{1, \ldots, k\}$ of data point $r$ is an unobserved random variable with $\mathbb{P}\left(I^{(r)} = c\right) = p_{c}$. Given $I^{(r)} = c$, the feature vector ${\mathbf x}^{(r)}$ is drawn from a multivariate normal distribution with mean ${\boldsymbol \mu}^{(c)}$ and covariance matrix $\mathbf{C}^{(c)}$. The degree of belonging is then the posterior distribution of the cluster index given the observed feature vector,

\begin{equation}
\tag{3}
u_{c}^{(r)} :=
\mathbb{P}\left(I^{(r)} = c \mid {\mathbf x}^{(r)}\right) =
\frac{p_{c}\,
\mathcal{N}\left({\boldsymbol \mu}^{(c)},\mathbf{C}^{(c)}\right)({\mathbf x}^{(r)})}
{\sum_{c'=1}^{k} p_{c'}\,
\mathcal{N}\left({\boldsymbol \mu}^{(c')},\mathbf{C}^{(c')}\right)({\mathbf x}^{(r)})}\text{,}
\end{equation}

the responsibility of cluster $c$ for the data point (Bishop, 2006, Sect. 9.2; Murphy, 2012, Sect. 11.2.3). The posterior degrees of belonging Eq. (3) are the probabilities of the $k$ values of one random variable, so they satisfy the simplex condition Eq. (1) by construction. The model parameters $p_{c}$, ${\boldsymbol \mu}^{(c)}$, and $\mathbf{C}^{(c)}$ of the Gaussian mixture model are learned by maximum likelihood. This is an instance of empirical risk minimization: the loss of a data point is the negative logarithm of the probability density function of the Gaussian mixture model at its feature vector, and the clustering error is the average of this loss over the dataset. For the days at Krems, a two-component Gaussian mixture model learned by the expectation–maximization algorithm has the means $(3.1, 12.3)$ and $(14.2, 27.5)$ and the cluster probabilities $0.63$ and $0.37$. Under it, $58$ of the $366$ days have no degree of belonging above $0.8$, $23$ of them in May and none between November and February (Fig. 2).

The degrees of belonging determine the clusters in turn. In the M-step of the expectation–maximization algorithm, the mean of cluster $c$ is the average of the feature vectors weighted by the degrees of belonging. The cluster probability is the average of the degrees of belonging themselves. In formulas,

\begin{equation}
\tag{4}
{\boldsymbol \mu}^{(c)} =
\frac{\sum_{r=1}^{m}
u_{c}^{(r)}\, {\mathbf x}^{(r)}}
{\sum_{r=1}^{m} u_{c}^{(r)}}
\qquad \text{and} \qquad
p_{c} = \frac{1}{m}
\sum_{r=1}^{m} u_{c}^{(r)}
\quad \text{for } c = 1, \ldots, k\text{.}
\end{equation}

The covariance matrix $\mathbf{C}^{(c)}$ is the covariance matrix of the feature vectors under the same weighting (Bishop, 2006, Sect. 9.2.2). A day with the degrees of belonging $0.50$ and $0.50$ counts half towards each mean. With the degrees of belonging Eq. (2) of a hard clustering, the weighted average Eq. (4) is the mean of the data points assigned to cluster $c$, which is the averaging step of Lloyd's algorithm for k-means, and $p_{c}$ is the share of data points in that cluster. The expectation–maximization algorithm alternates the posterior Eq. (3) with the weighted averages Eq. (4), so the degrees of belonging are recomputed in every iteration: they are not model parameters but a function of the model parameters and of the feature vector. At the fixed point of the expectation–maximization algorithm on the days at Krems, each mean equals the average of the days weighted by their degrees of belonging and differs from the k-means cluster centroid of the same cluster by at most $1.9$ degrees.

Hard clustering is also a limit of the posterior degrees of belonging Eq. (3). Fix every cluster probability at $1/k$ and every covariance matrix at $\sigma^{2} \mathbf{I}$, a multiple of the identity matrix $\mathbf{I}$ with one common variance $\sigma^{2}$. The posterior Eq. (3) then depends on ${\mathbf x}^{(r)}$ only through its squared Euclidean distances to the means. As $\sigma^{2}$ shrinks towards $0$, it concentrates on the nearest mean: every degree of belonging tends to $0$ or $1$, and the expectation–maximization algorithm reduces to Lloyd's algorithm (Bishop, 2006, Sect. 9.3.2; Hastie et al., 2009, Sect. 14.3.7). For the days at Krems, with the two k-means cluster centroids as means and $\sigma^{2} = 1$, no day has both degrees of belonging above $0.2$, and every degree of belonging is within $0.01$ of its k-means value Eq. (2). With $\sigma^{2} = 50$, $16$ percent of the days have both degrees above $0.2$. With $\sigma^{2} = 1000$, every day has both above $0.2$ and no degree of belonging exceeds $0.63$.

The degrees of belonging of a data point are the value of a hypothesis at its feature vector. The posterior Eq. (3) is a map $h: \mathbb{R}^{d} \rightarrow \Delta^{k-1}$ determined by the model parameters of the Gaussian mixture model, and the assignment rule of k-means is a map from $\mathbb{R}^{d}$ to the vertices of $\Delta^{k-1}$ determined by the cluster centroids. Either map applies to a day that did not belong to the dataset the clusters were learned from, which is how a clustering generalizes. A hard clustering is derived from degrees of belonging by assigning each data point to the cluster with the largest one, and $1 - \max_{c = 1, \ldots, k}
u_{c}^{(r)}$ measures how uncertain that assignment is (Murphy, 2012, Sect. 11.2.3). For the days at Krems, the largest posterior degree of belonging names the k-means cluster on $91.5$ percent of the days. In medical image segmentation, the degrees of belonging of a pixel to the tissue types let a pixel at the boundary between two tissues belong partly to both, where a hard clustering must pick one (Pham et al., 2000).

![Figure 2 of the entry dob](https://dictionaryofml.org/terms/fig/dob-fig2.svg)

*Figure 2: (a) Every $18$th of the $366$ days of $2024$ at Krems, $21$ days, drawn by their posterior degrees of belonging Eq. (3) under the two-component Gaussian mixture model: cold days (circles) and warm days (squares) have a degree of belonging above $0.8$ to one cluster, in-between days (triangles) have both below $0.8$. The crosses are the two means, and the ellipses are the level sets of the two multivariate normal distributions at one standard deviation: the cold one (solid ellipse) and the warm one (dashed ellipse). (b) The degree of belonging to the warm cluster of every day of the year: the posterior under the Gaussian mixture model (solid curve) and the $0$ or $1$ of k-means (dashed curve). Data generated by pythondemos/dob.py*

**Synonyms:** membership degree, responsibility.

**See also:** [cluster](https://dictionaryofml.org/terms/cluster.html), [clustering](https://dictionaryofml.org/terms/clustering.html), [soft clustering](https://dictionaryofml.org/terms/softclustering.html), [hard clustering](https://dictionaryofml.org/terms/hardclustering.html), [cluster centroid](https://dictionaryofml.org/terms/clustercentroid.html), [Gaussian mixture model](https://dictionaryofml.org/terms/gmm.html), [expectation–maximization](https://dictionaryofml.org/terms/em.html), [$k$-means](https://dictionaryofml.org/terms/kmeans.html), Lloyd's algorithm, posterior distribution, probability simplex, probability mass function, [clustering error](https://dictionaryofml.org/terms/clusteringerror.html).

## References

- Bezdek (1981). Pattern Recognition with Fuzzy Objective Function Algorithms. *Plenum Press*.
- Ruspini (1969). A new approach to clustering. *Information and Control*. <https://doi.org/10.1016/S0019-9958(69)90591-9>
- Krishnapuram and Keller (1993). A possibilistic approach to clustering. *IEEE Transactions on Fuzzy Systems*. <https://doi.org/10.1109/91.227387>
- Bishop (2006). Pattern Recognition and Machine Learning. *Springer Science+Business Media*. <https://doi.org/10.1007/978-0-387-45528-0>
- Murphy (2012). Machine Learning: A Probabilistic Perspective. *MIT Press*.
- Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. *Springer Science+Business Media*. <https://doi.org/10.1007/978-0-387-84858-7>
- Pham et al. (2000). Current methods in medical image segmentation. *Annual Review of Biomedical Engineering*. <https://doi.org/10.1146/annurev.bioeng.2.1.315>

## Cite this entry

```bibtex
@misc{dictml_dob,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {degree of belonging},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-20},
  url = {https://dictionaryofml.org/terms/dob.html}
}
```
