Dictionary of Applied Machine Learning

Gaussian random variable

Updated on 2026-09-18

Typeset PDF version — the authoritative form of this entry

Python demo — a script that recomputes what this entry states and prints one line per check

A standard Gaussian random variable (Gaussian RV) is a real-valued random variable (RV) with probability density function (pdf) proportional to $\exp(-\eta^{2}/2)$. A general Gaussian RV with mean $\mu$ and variance $\sigma^{2}$ is obtained by scaling and shifting a standard Gaussian RV. A Gaussian random vector is an affine transformation of independent standard Gaussian RVs. Gaussian RVs arise naturally as limits of averages of many independent RVs through the central limit theorem (CLT). Among all random vectors with a given covariance matrix, the Gaussian random vector maximizes the differential entropy.

Definition

P-stdA standard Gaussian random variable (RV) is a real-valued RV $\feature$ with a probability density function (pdf) (Bertsekas and Tsitsiklis, 2008; Gray, 2009; Papoulis and Pillai, 2002) \begin{equation} \nonumber \pdf{\feature}{\eta} = \frac{1}{\sqrt{2\pi}} \exp\,(-\eta^2/2) \text{.} \end{equation} Given a standard Gaussian RV $\feature$, a general Gaussian RV $\feature'$ with mean $\mu$ and variance $\sigma^2$ can be constructed via $\feature' \defeq \sigma \feature + \mu$. The probability distribution of a Gaussian RV is referred to as the normal distribution, denoted by $\mvnormal{\mu}{\sigma^{2}}$. Its pdf is \[ \pdf{\feature'}{\eta} = \frac{1}{\sqrt{2\pi\sigma^{2}}} \exp\Big( -\frac{(\eta-\mu)^{2}}{2\sigma^{2}} \Big) \text{.} \] Two methods generate Gaussian RVs. For exact generation, let $U$ be uniformly distributed on $(0,1)$ and let $\Phi$ be the cumulative distribution function (cdf) of a standard Gaussian RV. Then $\feature' \defeq \mu + \sigma\Phi^{-1}(U)$ has distribution $\mvnormal{\mu}{\sigma^{2}}$. For approximate generation, if $\feature^{(i)}$ are independent and identically distributed (i.i.d.) RVs with mean $0$ and finite, nonzero variance $\tau^{2}$, then \[ \frac{1}{\tau\sqrt{n}}\sum_{i=1}^{n}\feature^{(i)} \] converges in distribution to a standard Gaussian RV as $n$ increases. Thus, the central limit theorem (CLT) makes a Gaussian approximation appropriate for averages of many independent contributions, although a finite sum need not be Gaussian (Ross, 2014). The two parameters act separately on the pdf, as shown in Fig. 1. Adding $\mu$ shifts the pdf along the horizontal axis without changing its shape. For $\sigma>0$, multiplying the RV by $\sigma$ stretches the pdf horizontally by a factor of $\sigma$ and lowers its peak by the same factor, so its area remains $1$.

Figure 1 of the entry gaussrv
Figure 1: The pdf of a standard Gaussian RV $\feature$ and of the Gaussian RV $\feature' = \sigma \feature + \mu$ with $\mu = 2$ and $\sigma = 1.5$. The shift parameter $\mu$ moves the mean from $0$ to $\mu$, while the scale parameter $\sigma$ changes the standard deviation from $1$ to $\sigma$ and the variance from $1$ to $\sigma^{2}$
A Gaussian random vector $\featurevec \in \reals^{\featuredim}$ with covariance matrix $\covmtxgeneric$ and mean $\meanvecgeneric$ can be constructed as (Gray, 2009; Lapidoth, 2017; Papoulis and Pillai, 2002) \[ \featurevec \defeq \mA \vz + \meanvecgeneric, \] where $\vz \defeq \big( z_{1}, \,\ldots, \,z_{\featuredim} \big)^{\top}$ is a vector of i.i.d. standard Gaussian RVs, and $\mA \in \reals^{\featuredim \times \featuredim}$ satisfies $\mA \mA^{\top} = \covmtxgeneric$. The probability distribution of a Gaussian random vector is referred to as the multivariate normal distribution, denoted by $\mvnormal{\meanvecgeneric}{\covmtxgeneric}$. A Gaussian random vector need not have independent components; its components are independent exactly when its covariance matrix is diagonal.

P-processA Gaussian random vector $\featurevec=\big(\feature_{1},\,\ldots,\,\feature_{\featuredim}\big)^{\top}$ can be interpreted as a stochastic process indexed by the finite set $\{1,\,\ldots,\,\featuredim\}$. More generally, a Gaussian process (GP) is a stochastic process over an index set $\indexset$ whose restriction to every finite subset of $\indexset$ is a Gaussian random vector (Rasmussen and Williams, 2006).

Gaussian RVs are widely used probabilistic models in the statistical analysis of machine learning (ML) methods, including regression with additive Gaussian measurement noise. For example, a regression method can use $\truelabel = \weights^{\top}\featurevec + \epsilon$ with $\epsilon \sim \mvnormal{0}{\sigma^{2}}$. Maximizing the resulting likelihood is equivalent to minimizing squared loss, connecting the Gaussian noise assumption to empirical risk minimization (ERM).

P-maxentAmong all random vectors with a given covariance matrix $\covmtxgeneric$, the random vector $\featurevec \sim \mvnormal{\meanvecgeneric}{\covmtxgeneric}$ maximizes differential entropy (Cover and Thomas, 2006, Th. 8.6.5). It is therefore the least-committal choice of probability distribution when only the first two moments are fixed.

See also: multivariate normal distribution, Gaussian process, probabilistic model, central limit theorem, differential entropy.

References

  1. Bertsekas and Tsitsiklis (2008). Introduction to Probability. Athena Scientific.
  2. Gray (2009). Probability, Random Processes, and Ergodic Properties. Springer Science+Business Media. doi.org/10.1007/978-1-4419-1090-5
  3. Papoulis and Pillai (2002). Probability, Random Variables, and Stochastic Processes. McGraw-Hill Higher Education.
  4. Ross (2014). A First Course in Probability. Pearson Education.
  5. Lapidoth (2017). A Foundation in Digital Communication. Cambridge Univ. Press.
  6. Rasmussen and Williams (2006). Gaussian Processes for Machine Learning. MIT Press. doi.org/10.7551/mitpress/3206.001.0001
  7. Cover and Thomas (2006). Elements of Information Theory. Wiley. doi.org/10.1002/047174882X

Cite this entry

@misc{dictml_gaussrv,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {Gaussian random variable},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-09-21},
  url = {https://dictionaryofml.org/terms/gaussrv.html}
}