Dictionary of Applied Machine Learning: 115 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here

decision region

Updated on 2026-11-08

Readiness 80/100

▶ Run the Python demo Open in Colab Typeset PDF LaTeX source Markdown Cite this entry

See also classification binary classification decision boundary linear classifier decision tree logistic regression

A decision region of a classifier is the set of all feature vectors that the classifier maps to one particular label value. The decision regions partition the feature space, one region per label value, and all feature vectors in the same region obtain the same prediction, so the decision regions determine the classifier completely. The union of the boundaries of the decision regions is the decision boundary. The shape of the decision regions depends on the model: a linear classifier has halfspaces, a decision tree has unions of axis-parallel rectangles, and logistic regression on polynomial features has regions separated by a curve.

Figure produced by the Python demo of the entry decision region
Figure written by the entry's Python demo decisionregion.py, which recomputes what the entry states: the script block by block · open in Colab

Definition

B-partitionA frost warning predicts from the minimum and the maximum air temperature of a day whether the following night brings frost. Any rule that gives this warning splits the pairs of temperatures into two parts: the pairs for which it predicts frost and the pairs for which it does not. Each part is a decision region of the rule. In general, a classifier is a hypothesis $h\colon \mathcal{X} \rightarrow \mathcal{Y}$ with a finite label space $\mathcal{Y}$. Its decision region for the label value $\widehat{y} \in \mathcal{Y}$ is the set of feature vectors that it maps to this value, \[ \mathcal{R}_{\widehat{y}} := \big\{ {\mathbf x} \in \mathcal{X} : h({\mathbf x}) = \widehat{y} \big\} \text{.} \] The decision regions partition the feature space: every feature vector lies in exactly one of them, there is one region per label value, and a region may consist of several separate pieces (Bishop, 2006, Sect. 1.5.1). All feature vectors in the same decision region obtain the same prediction, so the decision regions determine the classifier completely. The boundary $\partial \mathcal{R}_{\widehat{y}}$ of a decision region is the set of feature vectors such that every neighborhood contains elements of $\mathcal{R}_{\widehat{y}}$ and of its complement. The union of these boundaries over all label values is the decision boundary.

B-shapeThe shape of the decision regions depends on the model. A linear classifier with $h({\mathbf x}) = \operatorname{sign}({\mathbf w}^{\top} {\mathbf x} + w_{0})$ has the two halfspaces on either side of the hyperplane ${\mathbf w}^{\top} {\mathbf x} + w_{0} = 0$ as its decision regions (Bishop, 2006, Sect. 4.1.1). For more than two label values, a linear classifier predicts the value with the largest of $K$ linear scores. Each decision region is then an intersection of $K - 1$ halfspaces and therefore convex (Bishop, 2006, Sect. 4.1.2). A decision tree compares one feature with a threshold at each split, so its decision regions are unions of axis-parallel rectangles (Hastie et al., 2009, Sect. 9.2.1). Logistic regression on polynomial features has decision regions separated by a curve, the level set of a polynomial.

B-determinedFig. 1 shows the decision regions of three classifiers learned from the same data points with two features: a linear classifier, a decision tree, and logistic regression on polynomial features. A data point whose label differs from the label value of the region containing its feature vector is misclassified, and the fraction of such data points is the average $0/1$ loss. The two label values are separated by a curve and not by any straight line, so the linear classifier misclassifies half of the data points, while the other two misclassify none.

Figure 1 of the entry decisionregion
Figure 1: Decision regions of three classifiers learned from the same ten data points with two features. Filled squares are data points with label $\truelabel = +1$, and open circles are data points with label $\truelabel = -1$. The shaded area is the decision region $\decreg{+1}$, the unshaded area the decision region $\decreg{-1}$, and the border between them the decision boundary. (a) A linear classifier, whose regions are two half-planes. (b) A decision tree of depth four, whose regions are unions of axis-parallel rectangles. (c) Logistic regression on all monomials of the two features up to degree two, whose regions are separated by a curve. Data generated by pythondemos/decisionregion.py

See also: classifier, classification, binary classification, decision boundary, boundary, neighborhood, linear classifier, decision tree, logistic regression, halfspace, hyperplane, feature space, label space, $\bf 0/1$ loss.

References

  1. Bishop (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media. doi.org/10.1007/978-0-387-45528-0
  2. Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science+Business Media. doi.org/10.1007/978-0-387-84858-7

Cite this entry

@misc{dictml_decisionregion,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {decision region},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-11},
  url = {https://dictionaryofml.org/terms/decisionregion.html}
}