# baseline

*Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-11-11. [HTML](https://dictionaryofml.org/terms/baseline.html) · [typeset PDF](https://dictionaryofml.org/terms/baseline.pdf) · [LaTeX](https://dictionaryofml.org/terms/baseline.tex) · [Python demo](https://dictionaryofml.org/terms/baseline.py) · [notebook](https://dictionaryofml.org/terms/baseline.ipynb)*

Math is LaTeX with the book's macros expanded, so no notation package is needed to read it.

## Abstract

A baseline is a reference level against which the performance of a trained model is compared. The cheapest baseline uses no features at all and predicts the same value for every data point, e.g., the sample mean of numeric label values, or the most frequent label value in classification. Any useful method must beat that level. Baselines can be obtained from human performance, from an existing machine learning method, or from a probabilistic model. Given an accurate probabilistic model of the data generation, the smallest achievable risk is the Bayes risk, incurred by the Bayes estimator of the label given the features. In practice, however, the Bayes estimator is infeasible for at least two reasons. First, the probability distribution of the data generation process might be impossible to determine. Second, computing the Bayes risk might be computationally too expensive.

## Definition

On two days of 2024, a weather station Krems an der Donau (Austria) recorded the same minimum temperature of $11.8\,^{\circ}\mathrm{C}$, but a maximum of $13.2\,^{\circ}\mathrm{C}$ on one day and $29.9\,^{\circ}\mathrm{C}$ on the other. No hypothesis that predicts the maximum daytime temperature from the minimum can be right on both days. Whatever loss a hypothesis incurs on those two days is unavoidable, so a measured loss above zero is not by itself a sign of a poor machine learning method.

More generally, consider a machine learning method that produces a learned hypothesis (or trained model) $\widehat{h} \in \mathcal{H}$. It is evaluated by computing the average loss on a test set (Hastie et al., 2009, Sect. 7.2). The question is whether the learned hypothesis is already close to optimal. If it is, little is gained by collecting more data or by spending more computation to improve it. Answering the question needs a baseline: a reference level for the average loss, either the smallest that any method can achieve or the level that any useful method must beat.

Sometimes a baseline is computed from the dataset itself, as in the scatterplot of Fig. 1 (Hastie et al., 2009, Sect. 2.4). The figure shows weather records of the station Krems an der Donau (Austria). Each of the 366 days of 2024 is a data point whose feature $x$ is the minimum air temperature of the day and whose label $y$ is its maximum air temperature. For the 15 data points whose $x$ is between $11.5\,^{\circ}\mathrm{C}$ and $12.5\,^{\circ}\mathrm{C}$, the corresponding label ranged from $13.2\,^{\circ}\mathrm{C}$ to $33.4\,^{\circ}\mathrm{C}$. Two of these data points even share the same feature value $x = 11.8$, with labels $13.2$ and $29.9$. A hypothesis is a map, so it assigns both days the same prediction. On these two data points it therefore incurs an average squared error loss of at least $(16.7/2)^{2} \approx 69.7$. The same holds wherever data points share a feature value: the spread of their labels determines an average loss that no hypothesis, however learned, can avoid.

The gray strips of Fig. 1 collect data points with very similar feature values. Each strip holds at least four data points and is drawn over the range of labels those data points carry. A strip is therefore narrow where data points are dense along the feature axis and wide where they are sparse, and it is tall wherever those data points carry very different labels. Computing the sample variance of the labels within each strip and averaging the results across all strips yields $18.75$ (in squared degrees Celsius). The resulting value, $18.75$ squared degrees Celsius, is an estimate of the smallest average squared error loss any hypothesis can achieve on these data points (Hastie et al., 2009, Sect. 2.4). The hypothesis learned by linear regression, the solid line of Fig. 1, incurs an average squared error loss of $19.17$, its training error. That is within three percent of the baseline and evidence that little is left to gain.

![Figure 1 of the entry baseline](https://dictionaryofml.org/terms/fig/baseline-fig1.svg)

*Figure 1: Observations collected at the weather station Krems an der Donau during 2024. Each data point is one day, with the minimum temperature as its feature $x$ and the maximum temperature as its label $y$. The shaded gray strips provide the data-derived baseline of $18.75$ squared degrees Celsius. The gray strips cut the feature axis into pieces that each hold at least four data points, each drawn over the range of labels its data points carry; their widths differ because the data points are not spread evenly over the feature axis. The open triangles mark the $15$ data points of one such strip, the one around $x=12$, whose feature values lie in $[11.5,12.5)$, and the double arrow spans the labels those $15$ data points carry. The filled circles are $20$ of the $351$ data points outside that strip, every $18$th of them. The solid line is the hypothesis learned by linear regression. Averaged over all strips, a spread of this kind estimates the smallest achievable average loss for any machine learning method that predicts the maximum temperature solely from the minimum temperature. Data generated by pythondemos/baseline.py*

Baselines can provide lower bounds or upper bounds on the achievable average loss. The strip-wise estimate of Fig. 1 approximates a lower bound, the Bayes risk defined below: no hypothesis has a smaller risk.

An upper bound, i.e., a baseline that any useful method should beat, comes from a trivial hypothesis map that does not depend on the features at all. For numeric labels, such a trivial prediction could be the sample mean of the labels in the training set. In classification, the constant prediction could be the label value of the majority class in the training set. Its accuracy is the share of that class, which on imbalanced data is already high. For the temperature dataset of Fig. 1, the constant prediction by the sample mean incurs an average squared error loss of $84.64$, more than four times the above baseline value $18.75$.

A second source of a baseline is a probabilistic model, which provides a precise notion of a baseline that applies to any machine learning method. In particular, the Bayes risk is the risk of the optimal hypothesis: the Bayes estimator of the label $y$ of a data point, given its features ${\mathbf x}$ (Lehmann and Casella, 1998, Thm. 1.1, p. 228). For the squared error loss, the Bayes risk equals the expectation of the posterior variance, taken over the features (Bishop, 2006, Sect. 1.5.5, Eq. (1.90)). The strip-wise estimate in Fig. 1 approximates the Bayes risk by local averaging (Hastie et al., 2009, Sect. 2.4): the sample variance of the labels within a strip around feature value $x'$ approximates the conditional variance $\sigma^{2}_{y \mid x'}$. The average of the strip-wise sample variances approximates the expectation $\mathbb{E}\{\sigma^{2}_{y \mid x}\}$.

This probabilistic interpretation also motivates the constant baseline above. The constant prediction of the majority class is itself a Bayes estimator for a probabilistic model $\mathbb{P}^{(y)}$ that ignores the features of data points. Under the 0/1 loss, the Bayes estimator assigns the label with the largest posterior probability (Bishop, 2006, Sect. 1.5.1). For a probabilistic model that does not consider features, the posterior distribution equals the prior distribution of the label and, in turn, the Bayes estimator is the most probable label. The Bayes risk is then the probability that the label takes a value that is different from the most likely one.

For a given choice of loss function, the Bayes estimator (if it exists) is fully determined by the probability distribution $\mathbb{P}$ (Lehmann and Casella, 1998, Thm. 1.1, p. 228). However, computing the Bayes estimator and Bayes risk presents two main challenges. First, the probability distribution $\mathbb{P}$ is unknown and must be estimated from observed data (Hastie et al., 2009, Sect. 2.4). Second, even if $\mathbb{P}$ were known, exact probabilistic inference can be computationally infeasible for general probabilistic models, such as multiply connected Bayesian belief networks (Cooper, 1990, Sect. 4).

A widely-used probabilistic model for data points with numeric features and label is the multivariate normal distribution $\left( {\mathbf x},y \right) \sim \mathcal{N}\left({\boldsymbol \mu},{\mathbf C}\right)$. Here, for the squared error loss, the Bayes estimator is the posterior mean $\mu_{y|{\mathbf x}} := \mathbb{E}\{y \mid {\mathbf x}\}$ of the label $y$, given the features ${\mathbf x}$ (Gray, 2009; Lehmann and Casella, 1998, Cor. 1.2, p. 228). The corresponding Bayes risk is the posterior variance $\sigma^{2}_{y|{\mathbf x}}$ (see Fig. 2). It is the baseline a trained hypothesis $\widehat{h}$ is compared against. Fig. 2 shows its prediction $\widehat{h}({\mathbf x})$ beside the posterior mean.

![Figure 2 of the entry baseline](https://dictionaryofml.org/terms/fig/baseline-fig2.svg)

*Figure 2: The probability distribution (curve) of the label $y$ of a data point, given its features ${\mathbf x}$, when $\left( {\mathbf x},y \right)$ follows a multivariate normal distribution. The dashed line marks the posterior mean $\mu_{y|{\mathbf x}}$, the Bayes estimator under the squared error loss; the horizontal segment has length $\sigma_{y|{\mathbf x}}$, the square root of the posterior variance $\sigma^{2}_{y|{\mathbf x}}$ and hence of the Bayes risk; the cross marks the prediction $\widehat{h}({\mathbf x})$ of a trained hypothesis*

For the multivariate normal distribution, this posterior variance does not depend on ${\mathbf x}$ (Bishop, 2006, Sect. 2.3.1, Eq. (2.73)). More specifically, it is the reciprocal of the corresponding diagonal entry ${\mathbf C}^{-1}$, the inverse of the covariance matrix ${\mathbf C}$: $\sigma^{2}_{y|{\mathbf x}} = 1 / [{\mathbf C}^{-1}]_{y,y}$. In practice, the mean vector ${\boldsymbol \mu}$ and the covariance matrix ${\mathbf C}$ can be estimated from a dataset by the sample mean and the sample covariance matrix (Anderson, 2003, Thm. 3.2.1).

The baselines so far are derived from the data or from a probabilistic model. Three further approaches to constructing a baseline use existing methods for determining predictions. First, a baseline might be obtained from human performance, e.g., the misclassification rate of dermatologists who diagnose skin cancer from images (Salinas et al., 2024, Table 1). Second, for LLMs, baselines are often obtained from a benchmark, i.e., a fixed collection of test problems together with an evaluation protocol. As an example, the benchmark GSM8K measures the ability of a large language model to solve grade-school mathematics word problems (Cobbe et al., 2021). A third source is an existing machine learning method that is unsuitable for the intended machine learning application, e.g., because it is computationally too expensive. The test set error obtained from such a method can still serve as a baseline.

**See also:** Bayes risk, Bayes estimator, [test set](https://dictionaryofml.org/terms/testset.html), probabilistic model, risk, [accuracy](https://dictionaryofml.org/terms/accuracy.html), $\bf 0/1$ loss, imbalanced data, [confusion matrix](https://dictionaryofml.org/terms/cm.html), benchmark, multivariate normal distribution.

## References

- Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. *Springer Science+Business Media*. <https://doi.org/10.1007/978-0-387-84858-7>
- Lehmann and Casella (1998). Theory of Point Estimation. *Springer-Verlag*. <https://doi.org/10.1007/b98854>
- Bishop (2006). Pattern Recognition and Machine Learning. *Springer Science+Business Media*. <https://doi.org/10.1007/978-0-387-45528-0>
- Cooper (1990). The computational complexity of probabilistic inference using Bayesian belief networks. *Artif. Intell.*. <https://doi.org/10.1016/0004-3702(90)90060-D>
- Gray (2009). Probability, Random Processes, and Ergodic Properties. *Springer Science+Business Media*. <https://doi.org/10.1007/978-1-4419-1090-5>
- Anderson (2003). An Introduction to Multivariate Statistical Analysis. *Wiley*.
- Salinas et al. (2024). A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis. *npj Digit. Med.*. <https://doi.org/10.1038/s41746-024-01103-x>
- Cobbe et al. (2021). Training Verifiers to Solve Math Word Problems. <https://doi.org/10.48550/arXiv.2110.14168>

## Cite this entry

```bibtex
@misc{dictml_baseline,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {baseline},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-11},
  url = {https://dictionaryofml.org/terms/baseline.html}
}
```