Dictionary of Applied Machine Learning: 114 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here

Gaussian random variable (Gaussian RV)

Updated on 2026-10-29

Readiness 100/100

▶ Run the Python demo Open in Colab Typeset PDF LaTeX source Markdown Cite this entry

A Gaussian random variable (Gaussian RV) is a real-valued random variable (RV) whose probability density function (pdf) is proportional to the exponential of a negative quadratic. The pdf is determined by two numbers, the mean $\mu$ and the variance $\sigma^{2}$ of the RV; the probability distribution with these two parameters is denoted by $\mathcal{N}\left(\mu,\sigma^{2}\right)$. The standard Gaussian RV is the special case $\mu = 0$ and $\sigma^{2} = 1$; a general Gaussian RV is obtained from it by scaling and shifting. A Gaussian random vector is an affine transformation of independent standard Gaussian RVs. Gaussian RVs arise as limits of averages of many independent RVs through the central limit theorem (CLT). Among all random vectors with a given covariance matrix, the Gaussian random vector maximizes the differential entropy.

Figure produced by the Python demo of the entry Gaussian random variable (Gaussian RV)
Figure written by the entry's Python demo gaussrv.py, which recomputes what the entry states: the script block by block · open in Colab

Definition

P-defThe measurement error of a sensor, such as a thermometer, is usually modeled as a Gaussian random variable (RV). A real-valued RV $z$ is a standard Gaussian RV if its probability density function (pdf) is (Bertsekas and Tsitsiklis, 2008; Gray, 2009; Papoulis and Pillai, 2002) \[ p^{(z)}\left(\eta\right) = \frac{1}{\sqrt{2\pi}} \exp\,(-\eta^2/2) \text{.} \] Given a standard Gaussian RV $z$, a general Gaussian RV $x$ with mean $\mu$ and variance $\sigma^2$ can be constructed via $x := \sigma z + \mu$, so that the probability distribution of $x$ is the normal distribution $\mathcal{N}\left(\mu,\sigma^{2}\right)$, denoted by $x \sim \mathcal{N}\left(\mu,\sigma^{2}\right)$. The pdf of $x$ is, for every $\eta \in \mathbb{R}$, \[ p^{(x)}\left(\eta\right) = \frac{1}{\sqrt{2\pi\sigma^{2}}} \exp\Big( -\frac{(\eta-\mu)^{2}}{2\sigma^{2}} \Big) \text{.} \] The standard Gaussian RV is the special case $\mu = 0$ and $\sigma^{2} = 1$.

P-paramThe two parameters of $\mathcal{N}\left(\mu,\sigma^{2}\right)$ act separately on the pdf, as shown in Fig. 1: the parameter $\mu$ is the mean of $x$, and the parameter $\sigma$ is its standard deviation, the square root of the variance $\sigma^{2}$. Adding $\mu$ to the scaled RV $\sigma z$ shifts the pdf along the horizontal axis by $\mu$, without changing its shape. For $\sigma>0$, multiplying $z$ by $\sigma$ stretches the pdf horizontally by a factor of $\sigma$ and lowers its peak by the same factor, so the area under it remains $1$.

Figure 1 of the entry gaussrv
Figure 1: The solid curve is the pdf of a standard Gaussian RV $z$. The dashed curve is the pdf of the Gaussian RV $\feature = \sigma z + \mu$ with $\mu = 2$ and $\sigma = 1.5$. The shift parameter $\mu$ moves the mean from $0$ to $\mu$, while the scale parameter $\sigma$ changes the standard deviation from $1$ to $\sigma$ and the variance from $1$ to $\sigma^{2}$. A vertical ruler marks the mean of each curve, at $\eta = 0$ and at $\eta = \mu$, and each horizontal double arrow runs from the mean of its own curve to one standard deviation away, $1$ for the solid curve and $\sigma$ for the dashed one. The gray arrow from peak to peak marks the shift and the scaling that carry the one pdf to the other
Two ways to generate a Gaussian RV from other sources of randomness are in common use. For exact generation, let $U$ be an RV uniformly distributed on the open interval $(0,1) \subset \mathbb{R}$ and let $\Phi$ be the cumulative distribution function (cdf) of a standard Gaussian RV. Then $\mu + \sigma\Phi^{-1}(U) \sim \mathcal{N}\left(\mu,\sigma^{2}\right)$, the same probability distribution as $x$ but obtained from a different source of randomness.

P-vecFor approximate generation, let $m$ be a positive integer and let $v^{(1)},\,\ldots,\,v^{(m)}$ be independent and identically distributed (i.i.d.) RVs, not necessarily Gaussian, with mean $0$ and finite, nonzero variance $\tau^{2}$. Then the normalized sum \[ \frac{1}{\tau\sqrt{m}}\sum_{i=1}^{m}v^{(i)} \] converges in distribution to a standard Gaussian RV as $m$ increases. Thus, the central limit theorem (CLT) makes a Gaussian approximation appropriate for averages of many independent contributions, which is why the measurement error of a sensor is usually modeled as Gaussian, although a finite sum need not be Gaussian (Ross, 2014). Fig. 2 shows both halves of that statement for coin flips, each taking the values $+1$ and $-1$ with probability $1/2$. The probability mass function (pmf) of the normalized sum of $m = 8$ of them already follows the shape of the standard Gaussian pdf, and the sum still takes only $9$ values. Convergence in distribution is convergence of the cdfs, so the largest gap between the cdf of the normalized sum and that of a standard Gaussian RV measures how far the approximation still is. It is $0.137$ for $m = 8$ and $0.050$ for $m = 64$.

Figure 2 of the entry gaussrv
Figure 2: Here, $s$ is the sum of $\samplesize = 8$ i.i.d. RVs, each taking the values $+1$ and $-1$ with probability $1/2$ (coin flips). The stems carry the pmf of the normalized sum $s/\sqrt{\samplesize}$, which takes $9$ values spaced $2/\sqrt{\samplesize}$ apart, so the normalized sum is not Gaussian. The dashed curve is the probability a standard Gaussian RV puts on the cell of that width around each value, its pdf there times $2/\sqrt{\samplesize}$. Data generated by pythondemos/gaussrv.py
A Gaussian random vector ${\mathbf x} \in \mathbb{R}^{d}$ with covariance matrix ${\mathbf C}$ and mean ${\boldsymbol \mu}$ can be constructed as (Gray, 2009; Lapidoth, 2017; Papoulis and Pillai, 2002) \[ {\mathbf x} := {\mathbf A} {\mathbf z} + {\boldsymbol \mu} \text{.} \] Here, ${\mathbf z} := \big( z_{1}, \,\ldots, \,z_{d} \big)^{\top}$ is a vector of i.i.d. standard Gaussian RVs. The matrix ${\mathbf A} \in \mathbb{R}^{d \times d}$ is a matrix square root of ${\mathbf C}$, i.e., any matrix with ${\mathbf A} {\mathbf A}^{\top} = {\mathbf C}$ (a Cholesky factor, for instance). The probability distribution of a Gaussian random vector is referred to as the multivariate normal distribution, denoted by $\mathcal{N}\left({\boldsymbol \mu},{\mathbf C}\right)$. A Gaussian random vector need not have independent components; its components are independent exactly when its covariance matrix is diagonal.

P-gpA Gaussian random vector ${\mathbf x}=\big(x_{1},\,\ldots,\,x_{d}\big)^{\top}$ can be interpreted as a stochastic process indexed by the finite set $\{1,\,\ldots,\,d\}$. More generally, a Gaussian process (GP) is a stochastic process over an index set $\mathcal{I}$ whose restriction to every finite subset of $\mathcal{I}$ is a Gaussian random vector (Rasmussen and Williams, 2006).

P-concA standard Gaussian RV $z$ obeys the Gaussian tail bound $\mathbb{P}\left(|z| \geq t\right) \leq 2 \exp(-t^{2}/2)$ for every positive number $t$. That bound also controls functions of i.i.d. RVs. Consider a standard Gaussian random vector ${\mathbf z} \in \mathbb{R}^{d}$ and a function $f : \mathbb{R}^{d} \to \mathbb{R}$ that is Lipschitz with constant $L$, i.e., $\big| f({\mathbf z}) - f({\mathbf z}') \big| \leq L \Vert {{\mathbf z} - {\mathbf z}'} \Vert$ for every pair of vectors ${\mathbf z}, {\mathbf z}' \in \mathbb{R}^{d}$. The deviation of $f({\mathbf z})$ from its expectation is then a sub-Gaussian random variable (sub-Gaussian RV) with parameter at most $L$. For every positive number $t$, \[ \mathbb{P}\left(\big| f({\mathbf z}) - \mathbb{E}\big\{ f({\mathbf z}) \big\} \big| \geq t\right) \leq 2 \exp\Big( \frac{-t^{2}}{2L^{2}} \Big) \text{.} \] The number $d$ of arguments does not appear in this concentration inequality (Wainwright, 2019, Thm. 2.26): the bound is the same for two arguments and for a thousand. It constrains the tails of $f({\mathbf z})$ and does not state that $f({\mathbf z})$ has a normal distribution. The Euclidean norm is an example with $L = 1$: the length $\Vert {{\mathbf z}} \Vert$ has expectation at most $\sqrt{d}$ and deviates from it by more than $t$ with probability at most $2\exp(-t^{2}/2)$, however large $d$ is.

P-entAmong all random vectors with a given covariance matrix ${\mathbf C}$, the Gaussian random vector ${\mathbf x} \sim \mathcal{N}\left({\boldsymbol \mu},{\mathbf C}\right)$ maximizes the differential entropy (Cover and Thomas, 2006, Thm. 8.6.5). This property is one reason why additive Gaussian measurement noise is a widely used probabilistic model in the statistical analysis of machine learning (ML) methods: when only the variance of the noise is known, a Gaussian is the least restrictive assumption.

See also: multivariate normal distribution, Gaussian process, probabilistic model, central limit theorem, differential entropy, concentration inequality, sub-Gaussian random variable.

References

  1. Bertsekas and Tsitsiklis (2008). Introduction to Probability. Athena Scientific.
  2. Gray (2009). Probability, Random Processes, and Ergodic Properties. Springer Science+Business Media. doi.org/10.1007/978-1-4419-1090-5
  3. Papoulis and Pillai (2002). Probability, Random Variables, and Stochastic Processes. McGraw-Hill Higher Education.
  4. Ross (2014). A First Course in Probability. Pearson Education.
  5. Lapidoth (2017). A Foundation in Digital Communication. Cambridge Univ. Press.
  6. Rasmussen and Williams (2006). Gaussian Processes for Machine Learning. MIT Press. doi.org/10.7551/mitpress/3206.001.0001
  7. Wainwright (2019). High-Dimensional Statistics: A Non-Asymptotic Viewpoint. Cambridge Univ. Press.
  8. Cover and Thomas (2006). Elements of Information Theory. Wiley. doi.org/10.1002/047174882X

Cite this entry

@misc{dictml_gaussrv,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {Gaussian random variable (Gaussian RV)},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-04},
  url = {https://dictionaryofml.org/terms/gaussrv.html}
}