variance.py, which recomputes what the entry states: the script block by block · open in ColabDictionary of Applied Machine Learning: 114 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here
Updated on 2026-10-28
Readiness 100/100
See also random variable mean probability distribution empirical risk minimization baseline covariance matrix
The variance of a real-valued random variable (RV) is the expectation of the squared difference between the RV and its mean. It quantifies the spread of the probability distribution of the RV around the mean: a small variance means that realizations concentrate near the mean. The term also refers to the sample variance of a dataset; the two usages are consistent, since the sample variance is the variance of the RV obtained by drawing a data point uniformly from the dataset. Under the squared error loss, the variance is the smallest risk that a constant prediction of a numeric label can achieve. For a random vector, the variance is the expectation of the squared Euclidean norm of the deviation from the mean. This quantity equals the trace of the covariance matrix of the random vector, i.e., the sum of the variances of its entries.
variance.py, which recomputes what the entry states: the script block by block · open in ColabThe Helsinki Kaisaniemi weather station recorded seven daily maximum temperatures between $19.6$ and $22.2\,^{\circ}$C on 24–30 August 2026 (Finnish Meteorological Institute, 2026), and every reading lies within two degrees of their average of $20.5\,^{\circ}$C. How far readings scatter around their average is what the variance measures.
P-defThe variance of a real-valued random variable (RV) $y$
is the expectation $\mathbb{E}\big\{ \big( y - \mathbb{E}\{y \} \big)^{2} \big\}$ of
the squared difference between $y$ and its
expectation $\mathbb{E}\{y \}$, also called its
mean.
The term is also used to refer to the sample variance of a
finite dataset
$\mathcal{D} = \big\{ y^{(1)}, \,\ldots, \,y^{(m)} \in \mathbb{R} \big\}$.
These two usages are consistent. A dataset defines a
discrete random variable (discrete RV) $\widetilde{y}^{(\mathcal{D})} = y^{(I)}$
on the sample space $\{1, \,\ldots, \,m\}$. Here, the
index $I$ is chosen uniformly at random, i.e.,
$\mathbb{P}\left(I=r\right)=1/m$ for all
$r=1,\ldots,m$, so the probability distribution of
$\widetilde{y}^{(\mathcal{D})}$ is the
empirical distribution of the dataset. The variance of
$\widetilde{y}^{(\mathcal{D})}$ is precisely the
sample variance
\[
\frac{1}{m} \sum_{r=1}^{m}
\Big( y^{(r)} - \frac{1}{m}
\sum_{r'=1}^{m} y^{(r')}
\Big)^{2} \text{,}
\]
i.e., the average squared deviation of the data points from
their sample mean.
Fig. 1 depicts the seven Helsinki readings
of 24–30 August 2026 as realizations of a real-valued
RV, together
with their sample mean and the deviation of each measurement
from it.
pythondemos/variance.py
P-minFor an RV with a finite second moment, i.e., $\mathbb{E}\{ y^{2} \} < \infty$, the mean is the constant $b$ that minimizes the risk $\mathbb{E}\{ ( y - b )^{2} \}$. The variance is the smallest value this risk attains, i.e., its minimum at $b = \mathbb{E}\{y\}$ (Bertsekas and Tsitsiklis, 2008): \[ \mathbb{E}\big\{ \big( y - \mathbb{E}\{y\} \big)^{2} \big\} = \min_{b \in \mathbb{R}} \mathbb{E}\big\{ ( y - b )^{2} \big\} \text{.} \]
P-ermThis minimum-risk characterization applies directly to the simplest machine learning (ML) problem, predicting a numeric label $y$ without using any features. The hypothesis space for this problem consists of the constant maps $h(\cdot) = b$ with $b \in \mathbb{R}$. Each such hypothesis amounts to a single bias term: it delivers the same prediction $b$ for every data point. Given a training set $\mathcal{D}^{(\rm train)} = \big\{ y^{(1)}, \,\ldots, \,y^{(m)} \big\}$ that consists of labels $y^{(r)} \in \mathbb{R}$ only, empirical risk minimization (ERM) with the squared error loss amounts to finding the constant that best predicts the label of a data point. The solution $\widehat{b}$ is the sample mean of the labels (see mean), and the training error of the learned hypothesis is the sample variance of the labels (Jung, 2022). For the temperatures of Fig. 1, the learned hypothesis is the dashed line: $\widehat{b} = 20.5\,^{\circ}$C, with training error equal to the sample variance $0.91$ computed above. The variance of the label therefore serves as a baseline: an ML method that uses features is useful only if its risk falls below the variance of the label.
P-vectorVariance extends from a real-valued RV to a random vector ${\mathbf x}$ as the expectation of the squared Euclidean norm of the deviation from the mean, $\mathbb{E}\big\{ \left\lVert {\mathbf x} - \mathbb{E}\{{\mathbf x} \} \right\rVert_{2}^{2} \big\} = {\rm tr} \left(\mathbf{C}^{({\mathbf x})}\right)$. This is the sum of the variances of the entries of ${\mathbf x}$, written compactly as the trace of the covariance matrix $\mathbf{C}^{({\mathbf x})}$, whose diagonal carries those variances (Murphy, 2012, Sect. 2.5.1).
Synonyms: second central moment.
See also: random variable, random vector, expectation, mean, sample variance, standard deviation, probability distribution, empirical risk minimization, squared error loss, baseline, covariance matrix, trace, outlier.
@misc{dictml_variance,
author = {Jung, Alexander},
editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
title = {variance},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-04},
url = {https://dictionaryofml.org/terms/variance.html}
}