Dictionary of Applied Machine Learning: 114 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here

autoencoder

Updated on 2026-10-31

Readiness 100/100

▶ Run the Python demo Open in Colab Typeset PDF LaTeX source Markdown Cite this entry

See also feature learning dimensionality reduction principal component analysis empirical risk minimization self-supervised learning transfer learning artificial neural network

An autoencoder is a machine learning (ML) method that learns an encoder and a decoder together. The encoder maps the feature vector of a data point to a new feature vector, referred to as code. The decoder reconstructs the original feature vector from the code. The code delivered by the encoder is typically much shorter than the original feature vector, so the code can retain only part of the information in the feature vector. Different autoencoders use different choices for the hypothesis spaces from which the encoder and the decoder are chosen. Principal component analysis (PCA) is obtained as the special case where both are linear maps, and a deep autoencoder uses a deep net on each side.

Figure produced by the Python demo of the entry autoencoder
Figure written by the entry's Python demo autoencoder.py, which recomputes what the entry states: the script block by block · open in Colab

Definition

B-linearThree hundred points along a curve in the plane (Fig. 3) can each be stored as a single number from which the point is rebuilt almost exactly, and an image can likewise be stored as a short code from which a close copy is rebuilt. An autoencoder learns an encoder $h$ from a hypothesis space $\mathcal{H}$ along with a decoder $h^{*}$ from a second hypothesis space $\mathcal{H}^{*}$ (Goodfellow et al., 2016, Ch. 14). The encoder maps the feature vector ${\mathbf x} \in \mathbb{R}^{d}$ of a data point to a code ${\mathbf z} = h({\mathbf x}) \in \mathbb{R}^{d'}$. The decoder maps the code back to a reconstruction $\widehat{{\mathbf x}} = h^{*}({\mathbf z})$ of the original feature vector (see Fig. 1). The encoder and the decoder can be learnt via empirical risk minimization (ERM) with a loss that measures the reconstruction error ${\mathbf x} - h^{*}(h({\mathbf x}))$, such as the squared error loss (Goodfellow et al., 2016, Sect. 14.1), \begin{equation} \label{eq:autoencoder_erm} \min_{h \in \mathcal{H},\; h^{*} \in \mathcal{H}^{*}} \frac{1}{m} \sum_{r=1}^{m} L\Big( {\mathbf x}^{(r)},\, h^{*}\big( h\big( {\mathbf x}^{(r)} \big) \big) \Big) \text{,} \end{equation} The training set needs no labels: what the decoder has to reproduce is the data point itself, so an autoencoder is a form of self-supervised learning, as it does not require manual labeling.

Figure 1 of the entry autoencoder
Figure 1: An autoencoder: the encoder $\hypothesis$ maps a feature vector $\featurevec$ to a code $\vz$, and the decoder $\hypothesis^{*}$ maps the code back to a reconstruction $\widehat{\featurevec}$. Both are learned jointly, by minimizing a quantitative measure of the reconstruction error $\featurevec - \hypothesis^{*}(\hypothesis(\featurevec))$
The length $d'$ of the code is typically much smaller than the length $d$ of the original feature vector, in which case the autoencoder is called undercomplete (Goodfellow et al., 2016, Sect. 14.1). The encoder then implements a form of compression that still allows the decoder to reconstruct the original feature vector with a small reconstruction error (Hinton and Salakhutdinov, 2006). The encoder map can afterwards be used as a feature transformation for learning tasks based on the same data points, which makes the autoencoder a feature learning and a dimensionality reduction method (Bengio et al., 2013). The autoencoder is rarely used for the reconstruction itself: the decoder is dropped and the code kept, so that a later learning task learns a hypothesis with few model parameters on the code rather than on the raw features (see transfer learning).

B-nonlinearDifferent autoencoder methods are obtained by different choices for the hypothesis spaces $\mathcal{H}$, $\mathcal{H}^{*}$ and the loss function in \(\eqref{eq:autoencoder_erm}\). The special case of linear maps for the encoder and the decoder, paired with the squared error loss, coincides with principal component analysis (PCA) (Goodfellow et al., 2016, Sect. 14.1). For some applications, nonlinear encoder and decoder maps represented by artificial neural networks (ANNs) are more useful, and a deep autoencoder uses a deep net on each side (Fig. 2). On the curve of Fig. 3, a code of one number and one hidden layer on each side leave an average squared error loss of $0.003$, against $0.457$ for the best linear map with the same code. The gain is not the ANNs as such but the shape of the reconstruction: linear maps with a code of one number can only place the reconstructions on a straight line, and these data points do not lie on one.

Figure 2 of the entry autoencoder
Figure 2: A deep autoencoder. The encoder $\hypothesis$ is a deep net of three layers that maps the feature vector $\featurevec$, here $\nrfeatures = 6$ numbers, to a code $\vz$ of $\nrfeatures' = 2$ numbers (shaded). The decoder $\hypothesis^{*}$ rebuilds the feature vector from that code with two further layers. The code is the narrowest layer, so everything the reconstruction uses has to pass through two numbers
Figure 3 of the entry autoencoder
Figure 3: Three hundred data points along a curve, compressed to a code of one number and rebuilt; the dots are $20$ of them, every $15$th. Linear maps place every reconstruction on a straight line (dashed line), which no choice of line brings close to these data points; ANNs follow the curve (solid curve), apart from a small loop near $\feature_{1} = 0.6$ that is a defect of the learned decoder, and leave an average squared error loss $150$ times smaller. Data generated by pythondemos/autoencoder.py

See also: encoder, decoder, feature learning, feature transformation, dimensionality reduction, principal component analysis, empirical risk minimization, squared error loss, self-supervised learning, transfer learning, artificial neural network, deep net.

References

  1. Goodfellow et al. (2016). Deep Learning. MIT Press.
  2. Hinton and Salakhutdinov (2006). Reducing the dimensionality of data with neural networks. Science. doi.org/10.1126/science.1127647
  3. Bengio et al. (2013). Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell.. doi.org/10.1109/TPAMI.2013.50

Cite this entry

@misc{dictml_autoencoder,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {autoencoder},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-03},
  url = {https://dictionaryofml.org/terms/autoencoder.html}
}