# supervised learning *Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-10-28. [HTML](https://dictionaryofml.org/terms/supervisedlearning.html) · [typeset PDF](https://dictionaryofml.org/terms/supervisedlearning.pdf) · [LaTeX](https://dictionaryofml.org/terms/supervisedlearning.tex) · [Python demo](https://dictionaryofml.org/terms/supervisedlearning.py) · [notebook](https://dictionaryofml.org/terms/supervisedlearning.ipynb)* Math is LaTeX with the book's macros expanded, so no notation package is needed to read it. ## Abstract Supervised learning is the machine learning setting that needs the label values of some data points. That requirement is what sets it apart from other forms of machine learning, such as unsupervised learning or reinforcement learning. Two main types of supervised learning are regression, for a numeric label, and classification, for a label with finitely many values. Empirical risk minimization learns a hypothesis of a hypothesis space that minimizes the average loss on a training set of labeled data points. A label can be obtained from a sensing device, computed, or annotated by a person. These labeling processes typically involve errors, so a recorded label value is a noisy estimate of the true one. ## Definition A weather station of GeoSphere Austria at Krems an der Donau records the minimum and the maximum temperature of each day (GeoSphere Austria, 2026). From ten such days, two learning tasks can be built: predicting the maximum temperature of a day from its minimum, and predicting whether the next day is at least two degrees warmer. Both need, for each of the ten days, the value to be predicted, its label. Supervised learning refers to machine learning methods that need the label values of some data points, the ones forming a training set. That requirement sets it apart from the other settings: in unsupervised learning, no such value is available, and the loss is defined through model parameters that summarize the whole dataset; in reinforcement learning, a reward signal measures the usefulness of a prediction instead (Sutton and Barto, 2018, Sect. 1.1). Supervised learning methods evaluate the quality of a hypothesis by comparing its prediction against a known label value (Sutton and Barto, 2018, Sect. 1.1; Hastie et al., 2009, Ch. 2). Empirical risk minimization learns a hypothesis by comparing its predictions for the data points in a training set with their actual labels. The training set $\mathcal{D}^{(\rm train)} = \big\{ \left( {\mathbf x}^{(r)},y^{(r)} \right) \big\}_{r=1}^{m}$ consists of $m$ labeled data points, each with a feature vector ${\mathbf x}^{(r)}$ and a label $y^{(r)}$. Among the hypotheses of a hypothesis space $\mathcal{H}$ it chooses one that minimizes the average loss on the training set, $$ \widehat{h} \in \mathop{\mathrm{arg\,min}}_{h \in \mathcal{H}} \frac{1}{m} \sum_{r=1}^{m} L\left(\left( {\mathbf x}^{(r)},y^{(r)} \right),h \right) \text{.} $$ The loss of a data point can be evaluated on that data point alone, since its label supplies the value to compare with. The label space $\mathcal{Y}$, the set of values a label can take, decides which of two main types a learning task is: $\mathcal{Y} = \mathbb{R}$ makes it regression, a discrete label space makes it classification. Both learning tasks can be built from the same data points. Fig. 1 shows two supervised learning tasks, one regression and one classification, obtained from the same data points, the ten days of Krems an der Donau. With the maximum temperature of the day as label, the learning task is regression, and empirical risk minimization with the squared error loss learns the curve of panel (a). The second learning task uses both temperatures of the day as features. Its label is $+1$ if the next day is at least two degrees warmer and $-1$ otherwise, so the learning task is classification. The two classes overlap, so no straight line separates them: the same morning can be followed by a warmer or by a cooler day. ![Figure 1 of the entry supervisedlearning](https://dictionaryofml.org/terms/fig/supervisedlearning-fig1.svg) *Figure 1: Ten data points representing daily weather conditions at Krems an der Donau, the 25th of each month from January to October, are used in two different supervised learning tasks. In (a) the feature of a day is its minimum temperature and the label is its maximum temperature (open circles), so the learning task is regression; the curve is a hypothesis learned by empirical risk minimization with the squared error loss. In (b) both temperatures of a day are features and the label is $+1$ if the next day is at least two degrees warmer (filled circles) and $-1$ otherwise (open squares), so the learning task is classification. Data generated by pythondemos/supervisedlearning.py* The label values of data points can be obtained in different ways, such as physical sensing devices, computations, or human annotation. For example, the maximum temperature of a day at Krems can be read from a calibrated thermometer of the weather station (GeoSphere Austria, 2026). The label value can also be obtained by numerical computations. GeoSphere Austria runs INCA, which computes the temperature across Austria on a grid of one kilometer (Haiden et al., 2010, Sect. 2; Haiden et al., 2011). INCA starts from the forecast of a numerical weather prediction model and corrects it with the differences between forecast and observation at the surrounding stations, so it returns a temperature value also for a location that has no station (Haiden et al., 2010, Sect. 5.1). The temperature recordings can be annotated by a person instead, a meteorologist or a panel of local residents. These labeling processes differ in the amount of required resources and in the resulting label quality. Each carries an error of its own, so a recorded label value is a noisy estimate of the true one. The station thermometer has a calibration tolerance. A computed label inherits the error of the equations and of the readings they were given. A resident might misreport temperatures. Human annotation also limits how large the training set can be, since it requires the expert whose judgment is to be reproduced. Crowdsourcing lifts this limit: ImageNet, a dataset of $3.2$ million annotated images at the time of its first publication, was assembled by annotators working through a crowdsourcing platform (Deng et al., 2009, Sect. 3.2). An annotated label records a judgment rather than a measurement, and two annotators need not return the same value. ImageNet was assembled to limit this disagreement. Several annotators vote on each image, which is kept only on a convincing majority. The number of votes required is fixed per category, from an initial subset of its images (Deng et al., 2009, Sect. 3.2). **See also:** [label](https://dictionaryofml.org/terms/label.html), labeled data point, [training set](https://dictionaryofml.org/terms/trainset.html), [empirical risk minimization](https://dictionaryofml.org/terms/erm.html), [hypothesis](https://dictionaryofml.org/terms/hypothesis.html), [loss](https://dictionaryofml.org/terms/loss.html), squared error loss, [classification](https://dictionaryofml.org/terms/classification.html), regression, classifier, [validation error](https://dictionaryofml.org/terms/valerr.html), [unsupervised learning](https://dictionaryofml.org/terms/unsupervisedlearning.html), [self-supervised learning](https://dictionaryofml.org/terms/selfsupervisedlearning.html), semi-supervised learning, [reinforcement learning](https://dictionaryofml.org/terms/reinforcementlearning.html). ## References - GeoSphere Austria (2026). GeoSphere Austria climate station archive. - Sutton and Barto (2018). Reinforcement Learning: An Introduction. *MIT Press*. - Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. *Springer Science+Business Media*. - Haiden et al. (2010). Integrated Nowcasting through Comprehensive Analysis (INCA) -- System description. - Haiden et al. (2011). The Integrated Nowcasting through Comprehensive Analysis (INCA) system and its validation over the Eastern Alpine region. *Weather and Forecasting*. - Deng et al. (2009). ImageNet: A large-scale hierarchical image database. *2009 IEEE Conference on Computer Vision and Pattern Recognition*. ## Cite this entry ```bibtex @misc{dictml_supervisedlearning, author = {Jung, Alexander}, editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard}, title = {supervised learning}, howpublished = {Dictionary of Applied Machine Learning (course edition)}, year = {2026}, doi = {10.5281/zenodo.21569296}, note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-04}, url = {https://dictionaryofml.org/terms/supervisedlearning.html} } ```