Dictionary of Applied Machine Learning: 107 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here

baseline

Updated on 2026-10-22

Readiness 100/100

▶ Run the Python demo Open in Colab Typeset PDF LaTeX source Cite this entry

See also test set accuracy confusion matrix

A baseline is a reference level against which the performance of a trained model is compared. The cheapest baseline uses no features at all and predicts the same value for every data point, e.g., the sample mean of numeric label values, or the most frequent label in classification. Any useful method must beat that level. Baselines can be obtained from human performance, from an existing machine learning (ML) method, or from a probabilistic model. Given an accurate probabilistic model of the data generation, the smallest achievable risk is the Bayes risk, incurred by the Bayes estimator of the label given the features. In practice, however, the Bayes estimator is infeasible for at least two reasons. First, the probability distribution of the data generation process might be impossible to determine. Second, computing the Bayes risk might be computationally too expensive.

Figure produced by the Python demo of the entry baseline
Figure written by the entry's Python demo baseline.py, which recomputes what the entry states: the script block by block · open in Colab

Definition

Consider some machine learning (ML) method that produces a learned hypothesis (or trained model) $\learnthypothesis \in \hypospace$. Its quality is typically evaluated by computing the average loss on a test set. But how can it be verified that the measured performance is satisfactory? The question is whether the learned hypothesis is already so close to optimal that little is gained by investing more resources, for data collection or for computation, to improve it. Answering it needs a baseline, a reference level for the smallest loss achievable by any method.

B-spreadSometimes a baseline is computed directly from a scatterplot such as Fig. 1. That figure shows weather records of the station Krems an der Donau. Each of the 366 days of 2024 is a data point whose feature $\feature$ is the minimum air temperature of the day and whose label $\truelabel$ is its maximum air temperature. On the 15 days with $\feature$ between $11.5$ and $12.5$ degrees Celsius, the label ranged from $13.2$ to $33.4$ degrees Celsius. Two of these days even share the same feature value $\feature = 11.8$, with labels $13.2$ and $29.9$. A hypothesis is a map, so it assigns both days the same prediction and incurs, on these two days, an average squared error loss of at least $(16.7/2)^{2} \approx 69.7$.

B-limitThe gray strips of Fig. 1 repeat this measurement across the whole feature axis. Each strip holds at least four days and is drawn over the range of labels those days carry. A strip is therefore wide where the days are sparse and narrow where they are dense. Averaging the variance of the label within the strips yields $18.75$ (in squared degrees Celsius) as an estimate for the smallest achievable average squared error loss. A hypothesis learned by linear regression incurs an average squared error loss of $19.17$ on the 366 days. That is above the estimate, as it must be, and within three percent of it: this hypothesis is already close to optimal.

Figure 1 of the entry baseline
Figure 1: Daily temperatures at the weather station Krems an der Donau during 2024. Each data point is one day, with the minimum temperature as its feature $\feature$ and the maximum temperature as its label $\truelabel$. Days with (nearly) the same feature value carry widely different label values; this spread limits the average loss achievable by any ML method that predicts the maximum temperature based on the minimum temperature. The gray strips cut the feature axis into pieces that each hold at least four days, each drawn over the range of labels its days carry; their widths differ because the days are not spread evenly over the feature axis. Data generated by pythondemos/baseline.py
Baselines can provide lower bounds or upper bounds on the achievable average loss. The strip-wise estimate of Fig. 1 is a lower bound: no method can do better.

B-trivialAn upper bound, i.e., a baseline that any useful method should beat, comes from a trivial hypothesis map that does not depend on the features at all. For numeric labels, such a trivial prediction could be the sample mean of the labels in the training set. In classification, the constant prediction could be the label value of the majority class in the training set, which on imbalanced data already attains a high accuracy. For the temperature dataset of Fig. 1, the sample mean is $17.91$ degrees Celsius and incurs an average squared error loss of $84.64$, more than four times the $18.75$ that the strips leave achievable.

A baseline might be obtained from human performance, e.g., the misclassification rate of dermatologists who diagnose cancer from visual inspection of skin (Salinas et al., 2024). For large language models (LLMs), baselines are often obtained from a benchmark, i.e., a fixed collection of test problems together with an evaluation protocol. As an example, the benchmark GSM8K measures the ability of an LLM to solve grade-school mathematics word problems (Cobbe et al., 2021).

Another source for a baseline is an existing, but for some reason unsuitable, ML method: it might be computationally too expensive for the intended ML application, yet its test set error can still serve as a baseline. A more principled source of a baseline is a probabilistic model. Given an accurate probabilistic model $p(\featurevec,\truelabel)$ of the data generation, the minimum achievable risk among any hypotheses (not even required to belong to the hypothesis space $\hypospace$) can often be determined precisely (Lehmann and Casella, 1998).

This minimum achievable risk (referred to as the Bayes risk) is the risk of the Bayes estimator of the label $\truelabel$ of a data point, given its features $\featurevec$. For the squared error loss, the Bayes risk equals the expectation of the conditional variance of the label, given the features (Bishop, 2006, Sect. 1.5.5, Eq. (1.90)); the strip-wise estimate in Fig. 1 approximates exactly this quantity.

The constant prediction of the majority class is itself a Bayes estimator. If the features carry no information about the label, the Bayes estimator under the $0/1$ loss is the most probable label, and the Bayes risk is the probability of the other one. The trivial baseline is thus the Bayes risk of the probabilistic model in which the label does not depend on the features. It measures what is already known before any feature is read.

For a given choice of loss function, the Bayes estimator (if it exists) is fully determined by the probability distribution $\probdist$ (Lehmann and Casella, 1998, Chap. 4). However, computing the Bayes estimator and Bayes risk presents two main challenges. First, the probability distribution $\probdist$ is unknown and must be estimated from observed data. Second, even if $\probdist$ were known, computing the Bayes risk exactly may be computationally infeasible (Cooper, 1990, Sect. 4).

One probabilistic model makes both quantities explicit: the multivariate normal distribution $\pair{\featurevec}{\truelabel} \sim \mvnormal{\meanvecgeneric}{\covmtxgeneric}$, for data points $\pair{\featurevec}{\truelabel}$ generated by independent and identically distributed (i.i.d.) random variables (RVs) with numeric features and labels. Here, for the squared error loss, the Bayes estimator is given by the posterior mean $\mu_{\truelabel|\featurevec}$ of the label $\truelabel$, given the features $\featurevec$ (Gray, 2009; Lehmann and Casella, 1998). The corresponding Bayes risk is given by the posterior variance $\sigma^{2}_{\truelabel|\featurevec}$ (see Fig. 2). For the multivariate normal distribution this posterior variance does not depend on $\featurevec$, and it is the reciprocal of the diagonal entry of the inverse matrix $\covmtxgeneric^{-1}$ of the covariance matrix that corresponds to the label $\truelabel$ (Bishop, 2006, Sect. 2.3.1). In practice, the mean vector $\meanvecgeneric$ and the covariance matrix $\covmtxgeneric$ can be estimated from a dataset by the sample mean and the sample covariance matrix (Anderson, 2003).

Figure 2 of the entry baseline
Figure 2: For data points $\pair{\featurevec}{\truelabel}$ generated by i.i.d. RVs with a common multivariate normal distribution, the minimum risk (under squared error loss) is obtained by the Bayes estimator $\mu_{\truelabel|\featurevec}$ of the label $\truelabel$ based on its features $\featurevec$. The corresponding minimum risk is given by the posterior variance $\sigma^{2}_{\truelabel|\featurevec}$. This quantity can serve as a baseline for the average loss of a trained model $\learnthypothesis$

See also: Bayes risk, Bayes estimator, test set, probabilistic model, risk, accuracy, $\bf 0/1$ loss, imbalanced data, confusion matrix, benchmark.

References

  1. Salinas et al. (2024). A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis. npj Digit. Med.. doi.org/10.1038/s41746-024-01103-x
  2. Cobbe et al. (2021). Training Verifiers to Solve Math Word Problems. doi.org/10.48550/arXiv.2110.14168
  3. Lehmann and Casella (1998). Theory of Point Estimation. Springer-Verlag. doi.org/10.1007/b98854
  4. Bishop (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media. doi.org/10.1007/978-0-387-45528-0
  5. Cooper (1990). The computational complexity of probabilistic inference using Bayesian belief networks. Artif. Intell.. doi.org/10.1016/0004-3702(90)90060-D
  6. Gray (2009). Probability, Random Processes, and Ergodic Properties. Springer Science+Business Media. doi.org/10.1007/978-1-4419-1090-5
  7. Anderson (2003). An Introduction to Multivariate Statistical Analysis. Wiley.

Cite this entry

@misc{dictml_baseline,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {baseline},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-10-27},
  url = {https://dictionaryofml.org/terms/baseline.html}
}