Dictionary of Applied Machine Learning
Updated on 2026-08-27
Typeset PDF version — the authoritative form of this entry
Python demo — a script that recomputes what this entry states and prints one line per check
Self-supervised learning constructs the labels of a training set from the data points themselves. Some features of a data point are withheld and used as its label while the rest remain features, so what results is an ordinary supervised learning problem, solved by empirical risk minimization (ERM) on data that no annotator has touched. In natural language processing (NLP) the withheld feature is the next token, which is how a large language model (LLM) is trained. In computer vision it is a set of pixels, deleted from an image and predicted from the patches left visible.
P-constructAn open image database returns thousands of photographs for a keyword such as cat, and a web crawl returns billions of sentences. Neither comes with a label saying what should be predicted. A prediction task can be made from any one of them: hide the last word of a sentence, and the words before it become a question whose answer is already known. Self-supervised learning constructs labels this way, from the data point itself. (de Sa, 1993) uses the term for a classifier whose label for one sensory modality is derived from a co-occurring input in another.
P-nlpSome of the features of a data point are withheld and used as the label; the remaining ones stay as the features. The result is an ordinary supervised learning problem, solved by empirical risk minimization (ERM), and its training set needs nothing beyond the raw data. A single unlabeled collection yields as many training sets as there are ways of hiding part of a data point (see Fig. 1).
P-visionIn natural language processing (NLP) a data point is a stretch of text, its features are the tokens $v^{(1)}, \ldots, v^{(\contextlen)}$ and its label is the token $v^{(\contextlen+1)}$ that follows. Every position in every document is one labeled data point, so a corpus of $\samplesize$ tokens gives on the order of $\samplesize$ of them. This is how a large language model (LLM) is trained (Brown et al., 2020). Masking a token in the middle and predicting it from both sides withholds a different feature of the same data point (Devlin et al., 2019).
P-composeIn computer vision the withheld features are pixels: an image is split into patches, most of them are deleted, and the method predicts the missing pixel values from those that remain. (He et al., 2022) delete $75\%$ of the patches and still recover recognizable content. A deep net solving either task computes a composition $\learnthypothesis = s \circ \featuretrafo$, where the feature transformation $\featuretrafo \colon \inspace \to \reals^{\featuredim}$ sends a data point to a vector of activations and the map $s$ sends that vector to the withheld feature. The withheld part can be predicted only from what $\featuretrafo$ has kept: a next token cannot be recovered from a vector that has discarded the context, nor a deleted patch from one that has discarded its surroundings. Fitting the composition is therefore what shapes $\featuretrafo$, even though $\featuretrafo$ appears nowhere in the loss. (He et al., 2022, Sect. 3) name the two halves: their encoder is $\featuretrafo$ and sees only the patches left visible, while their decoder is $s$ and rebuilds the pixels from its output. What restricts $\featuretrafo$ there is the deletion of most of the input, not a narrow layer.
P-transferIn both domains it is $\featuretrafo$ that is carried over. The
map $s$ is dropped, and a small replacement for it is fitted on
the output of $\featuretrafo$ using the few labels the later
task does have (see transfer learning, foundation model).
Self-supervised learning is therefore one answer to the same shortage
that motivates semi-supervised learning (SSL): there, unlabeled data points assist a
label-scarce problem directly; here, they are turned into a
labeled problem of their own first.
@misc{dictml_selfsupervisedlearning,
author = {Jung, Alexander and Olioumtsevits, Konstantina and Schnoor, Ekkehard},
title = {self-supervised learning},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-09-11},
url = {https://dictionaryofml.org/terms/selfsupervisedlearning.html}
}