A membership inference attack aims at determining whether a
particular data point was used to train a machine learning (ML)
model. The basic idea is that a learned hypothesis
delivers more accurate predictions for data points
that are similar to those used for training. An adversary
queries the learned hypothesis and compares the loss
of its prediction against a threshold. Mitigation strategies
include regularization, which makes an ML method
generalize beyond the training set: the learned
hypothesis then behaves similarly for data points in
the training set and for those outside it, which makes it harder
for an adversary to guess the data points of the
training set.
Figure written by the entry's Python demo membershipinferenceattack.py, which recomputes what the entry states: the script block by block · open in Colab
Definition
A volunteer observer
at Krems an der Donau reports the minimum and the maximum
temperature for selected days. Each day is one data point
${\mathbf z} = \left( x,y \right)$, whose feature
$x$ is that day's minimum temperature and whose label
$y$ is that day's maximum temperature, and the reported
days form the training set $\mathcal{D}^{(\rm train)}$. A company learns a
hypothesis $\widehat{h}$ from $\mathcal{D}^{(\rm train)}$ to predict
the maximum temperature of a day from its minimum. The company
publishes
$\widehat{h}$, so anyone can query it at any minimum
temperature. The GeoSphere weather station at Krems publishes the
minimum and the maximum temperature of every day. Anyone with both
can try to determine whether the report of a specific day was used
for training. Since the reports come from one person, a yes
tells that the observer was around Krems an der Donau on that day.
Whether the observer was at Krems on a specific day is
personal data, even though a temperature is not
(European Parliament and Council of the European Union, 2016, Art. 4). This scenario is an example of a
membership inference attack: a privacy attack where an
adversary tries to determine whether a particular data point
was used to train a machine learning (ML) model (Shokri et al., 2017;
Vassilev et al., 2024, Sect. 2.4.2).
For a specific day, the adversary reads that day's $x$ and
$y$ from the station's record, computes the
prediction $\widehat{h}(x)$, and compares it
against $y$. A
loss $\big(y - \widehat{h}(x)\big)^{2}$
below a threshold indicates that the report of that day was used
for training.
B-defendFig. 1 carries this attack
out on the 31 days of October 2024 at Krems. One threshold labels
all 31 days correctly: the loss is zero on every day the
observer reported and never below $0.23\,^{\circ}$C$^{2}$ on any
other day. The days it declares reported form two blocks of
consecutive days, 7 to 13 and 20 to 26 October, which is when the
observer was there. The hypothesis behind the figure is
obtained by linear regression on the features
$x'_{j} = \cos\big((j-1) \pi u\big)$ for
$j = 1, \ldots, d$, with $u$ the minimum
temperature $x$ rescaled to $[0,1]$ over the month; for
$d = 14$, one feature per reported day,
least squares reproduces every reported day exactly.
Figure 1: The loss the published hypothesis incurs on
each day of October 2024 at Krems an der Donau, computed by an
adversary from the weather station's own record. Every day that
the observer did not report carries a solid stem, drawn from
zero to its loss and topped by a disc. The 14 days the
observer did report carry no stem at all: the squares resting
on the horizontal axis mark a loss of zero. The days
those squares mark are 7 to 13 and 20 to 26 October, the two
stays, so the losses say not only which
days were used for training but also when
the observer was at Krems. Data generated by
pythondemos/membershipinferenceattack.py
The number $d$ sets the effective dimension of the underlying
hypothesis space and so how far the ML method can overfit.
Fig. 2 plots how much the
attack gains from that. Fix a threshold $\tau$ on the
loss. Of the 14
days the observer reported, the fraction incurring a loss of
at most $\tau$ is the detection rate. Of the 17 days of the month
the observer did not report, the fraction incurring a loss of
at most $\tau$ is the false-alarm rate. The advantage of the
attack is the largest value of the detection rate minus the
false-alarm rate over all thresholds. An adversary that answers yes
on every day, or no on every day, leaves the two rates equal and
the advantage zero, which is why the advantage measures what the
attack gains over guessing, and a threshold that separates
the two groups makes it one. The threshold is chosen with the answer already known, so the
advantage is the most any threshold could achieve rather than what
an adversary would obtain. With two features, it is $0.10$,
and it reaches one once $d$ reaches the number of
reported days.
Figure 2: How the attack of
Fig. 1 improves as the
hypothesis space grows. The curve is the advantage, the
largest difference over all thresholds between the fraction of
the 14 reported days whose loss falls below the threshold
and the fraction of the 17 other days whose loss does,
for a hypothesis learned from the same reported days with
$\featuredim$ features. Data generated by
pythondemos/membershipinferenceattack.py
The attack succeeds only to the extent that the learned
hypothesis incurs a smaller loss inside the
training set than outside it, which is what the generalization gap
measures. Under a probabilistic model for how the data points
and their membership of the training set arise, the success of
such an
attack is tied to that gap. The quantity is the same
difference as above, with the two rates replaced by
probabilities under that model. For a loss bounded by a
constant $B$, an adversary that answers yes with a
probability decreasing in the loss has a difference
that grows in proportion to the expected generalization gap
(Yeom et al., 2018, Thm. 2). Regularization
controls that gap: for a convexloss that is
Lipschitz with constant $L$, adding the
penalty term $\alpha \left\lVert {\mathbf w} \right\rVert_{2}^{2}$ bounds
the expected generalization gap by
$2L^{2}/(\alpha m)$
(Shalev-Shwartz and Ben-David, 2014, Cor. 13.6).
Fig. 3 repeats the
attack against the same features learned with that
penalty term, and the picture no longer tells the two groups
apart. Regularization therefore serves two purposes,
accurate predictions outside the training set
(Hastie et al., 2009, Sect. 7.4) and a defence
against a membership inference attack
(Shokri et al., 2017, Sect. VIII). For the observer at Krems, that is
the difference between a published hypothesis that reveals
nothing about them and one that reveals the days they were in town.
Figure 3: The same attack as in
Fig. 1, on the same axes,
against the hypothesis learned from the same 14 reported
days with the penalty term
$\regparam \normgeneric{\weights}{2}^{2}$ for
$\regparam = 10$. The reported days now carry dashed stems
topped by a square, of much the same height as the solid stems
topped by a disc that mark the days the observer did not
report, so no threshold picks them out.
Their average loss rises from $0$ to
$5.24\,^{\circ}$C$^{2}$ while the average loss elsewhere
falls from $14.29$ to $5.26\,^{\circ}$C$^{2}$: the
generalization gap closes to $0.02\,^{\circ}$C$^{2}$ and the
adversary's advantage drops from one to at most $0.15$. Data generated
by pythondemos/membershipinferenceattack.py
European Parliament and Council of the European Union (2016). Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (Text with EEA relevance). eur-lex.europa.eu/eli/reg/2016/679/oj
Shokri et al. (2017). Membership Inference Attacks Against Machine Learning Models. 2017 IEEE Symp. Secur. Privacy. doi.org/10.1109/SP.2017.41
Vassilev et al. (2024). Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. doi.org/10.6028/NIST.AI.100-2e2023
Yeom et al. (2018). Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting. 2018 IEEE 31st Computer Security Foundations Symposium (CSF). doi.org/10.1109/CSF.2018.00027
Shalev-Shwartz and Ben-David (2014). Understanding Machine Learning: From Theory to Algorithms. Cambridge Univ. Press. doi.org/10.1017/cbo9781107298019
Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science+Business Media. doi.org/10.1007/978-0-387-84858-7
Cite this entry
@misc{dictml_membershipinferenceattack,
author = {Jung, Alexander},
editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
title = {membership inference attack},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-13},
url = {https://dictionaryofml.org/terms/membershipinferenceattack.html}
}
