supervisedlearning.py, which recomputes what the entry states: the script block by block · open in ColabDictionary of Applied Machine Learning: 114 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here
Updated on 2026-10-28
Readiness 100/100
See also label training set empirical risk minimization hypothesis loss classification validation error unsupervised learning self-supervised learning reinforcement learning
Supervised learning is the machine learning (ML) setting that needs the label values of some data points. That requirement is what sets it apart from other forms of ML, such as unsupervised learning or reinforcement learning (RL). Two main types of supervised learning are regression, for a numeric label, and classification, for a label with finitely many values. Empirical risk minimization (ERM) learns a hypothesis of a hypothesis space that minimizes the average loss on a training set of labeled data points. A label can be obtained from a sensing device, computed, or annotated by a person. These labeling processes typically involve errors, so a recorded label value is a noisy estimate of the true one.
supervisedlearning.py, which recomputes what the entry states: the script block by block · open in ColabA weather station of GeoSphere Austria at Krems an der Donau records the minimum and the maximum temperature of each day (GeoSphere Austria, 2026). From ten such days, two learning tasks can be built: predicting the maximum temperature of a day from its minimum, and predicting whether the next day is at least two degrees warmer. Both need, for each of the ten days, the value to be predicted, its label. Supervised learning refers to machine learning (ML) methods that need the label values of some data points, the ones forming a training set. That requirement sets it apart from the other settings: in unsupervised learning, no such value is available, and the loss is defined through model parameters that summarize the whole dataset; in reinforcement learning (RL), a reward signal measures the usefulness of a prediction instead (Sutton and Barto, 2018, Sect. 1.1). Supervised learning methods evaluate the quality of a hypothesis by comparing its prediction against a known label value (Sutton and Barto, 2018, Sect. 1.1; Hastie et al., 2009, Ch. 2).
B-regressEmpirical risk minimization (ERM) learns a hypothesis by comparing its predictions for the data points in a training set with their actual labels. The training set $\mathcal{D}^{(\rm train)} = \big\{ \left( {\mathbf x}^{(r)},y^{(r)} \right) \big\}_{r=1}^{m}$ consists of $m$ labeled data points, each with a feature vector ${\mathbf x}^{(r)}$ and a label $y^{(r)}$. Among the hypotheses of a hypothesis space $\mathcal{H}$ it chooses one that minimizes the average loss on the training set, \[ \widehat{h} \in \mathop{\mathrm{arg\,min}}_{h \in \mathcal{H}} \frac{1}{m} \sum_{r=1}^{m} L\left(\left( {\mathbf x}^{(r)},y^{(r)} \right),h \right) \text{.} \] The loss of a data point can be evaluated on that data point alone, since its label supplies the value to compare with. The label space $\mathcal{Y}$, the set of values a label can take, decides which of two main types a learning task is: $\mathcal{Y} = \mathbb{R}$ makes it regression, a discrete label space makes it classification. Both learning tasks can be built from the same data points.
B-classifyFig. 1 shows two supervised
learning tasks, one regression and one
classification, obtained from the same data points, the
ten days of Krems an der Donau. With the maximum temperature of the day as
label, the learning task is regression, and
ERM with the squared error loss learns the curve of panel (a).
The second learning task uses both temperatures of the day as
features. Its label is $+1$ if the next day is at
least two degrees warmer and $-1$ otherwise, so the
learning task is classification. The two classes
overlap, so no straight line separates them: the same morning can
be followed by a warmer or by a cooler day.
pythondemos/supervisedlearning.py
These labeling processes differ in the amount of required resources and in the resulting label quality. Each carries an error of its own, so a recorded label value is a noisy estimate of the true one. The station thermometer has a calibration tolerance. A computed label inherits the error of the equations and of the readings they were given. A resident might misreport temperatures. Human annotation also limits how large the training set can be, since it requires the expert whose judgment is to be reproduced. Crowdsourcing lifts this limit: ImageNet, a dataset of $3.2$ million annotated images at the time of its first publication, was assembled by annotators working through a crowdsourcing platform (Deng et al., 2009, Sect. 3.2).
An annotated label records a judgment rather than a measurement, and two annotators need not return the same value. ImageNet was assembled to limit this disagreement. Several annotators vote on each image, which is kept only on a convincing majority. The number of votes required is fixed per category, from an initial subset of its images (Deng et al., 2009, Sect. 3.2).
See also: label, labeled data point, training set, empirical risk minimization, hypothesis, loss, squared error loss, classification, regression, classifier, validation error, unsupervised learning, self-supervised learning, semi-supervised learning, reinforcement learning.
@misc{dictml_supervisedlearning,
author = {Jung, Alexander},
editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
title = {supervised learning},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-04},
url = {https://dictionaryofml.org/terms/supervisedlearning.html}
}