# decision boundary

*Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-10-31. [HTML](https://dictionaryofml.org/terms/decisionboundary.html) · [typeset PDF](https://dictionaryofml.org/terms/decisionboundary.pdf) · [LaTeX](https://dictionaryofml.org/terms/decisionboundary.tex) · [Python demo](https://dictionaryofml.org/terms/decisionboundary.py) · [notebook](https://dictionaryofml.org/terms/decisionboundary.ipynb)*

Math is LaTeX with the book's macros expanded, so no notation package is needed to read it.

## Definition

Consider a hypothesis $h\colon\mathcal{X}\rightarrow\mathcal{Y}$ whose finite label space $\mathcal{Y}$ represents different classes. Its decision boundary consists of those feature vectors that are arbitrarily close to two or more different decision regions. More precisely, ${\mathbf x}\in\mathcal{X}$ lies on the decision boundary if every neighborhood of ${\mathbf x}$ contains feature vectors ${\mathbf x}'$ and ${\mathbf x}''$ such that $h({\mathbf x}')\neqh({\mathbf x}'')$. Thus, arbitrarily small perturbations near the decision boundary can result in different predicted labels (or classes).

A common construction of a classifier is by thresholding a real-valued scoring function $g\colon\mathcal{X}\rightarrow\mathbb{R}$. The classifier assigns different labels according to whether $g({\mathbf x})$ lies below or above a threshold $c$. If $g$ is continuous, its decision boundary is the level set $\{{\mathbf x}:g({\mathbf x})=c\}$, provided that $g$ takes values on both sides of $c$ arbitrarily close to every point in this set. For a linear classifier with $g({\mathbf x})={\mathbf w}^{\top}{\mathbf x}$ and ${\mathbf w}\neq\mathbf{0}$, this level set is the hyperplane ${\mathbf w}^{\top}{\mathbf x}=c$, whose normal vector is ${\mathbf w}$ (Bishop, 2006, Sect. 4.1; Hastie et al., 2009, Ch. 4).

Fig. 1 shows that the geometry of a decision boundary depends on the model. A linear classifier produces a straight boundary, a decision tree produces an axis-aligned, piecewise constant boundary, and logistic regression applied to polynomial features can produce a curved boundary.

![Figure 1 of the entry decisionboundary](https://dictionaryofml.org/terms/fig/decisionboundary-fig1.svg)

*Figure 1: Decision regions learned from the same ten-point training set by three classifiers. Gray marks the region assigned the label $+1$; the thick black lines are the learned decision boundaries. Open circles have label $-1$, and filled squares have label $+1$. The experiment is generated by pythondemos/decisionboundary.py*

**See also:** [hypothesis](https://dictionaryofml.org/terms/hypothesis.html), [feature vector](https://dictionaryofml.org/terms/featurevec.html), label space, boundary, decision region, neighborhood.

## References

- Bishop (2006). Pattern Recognition and Machine Learning. *Springer Science+Business Media*. <https://doi.org/10.1007/978-0-387-45528-0>
- Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. *Springer Science+Business Media*. <https://doi.org/10.1007/978-0-387-84858-7>

## Cite this entry

```bibtex
@misc{dictml_decisionboundary,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {decision boundary},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-02},
  url = {https://dictionaryofml.org/terms/decisionboundary.html}
}
```