Dictionary of Applied Machine Learning · membership inference attack
Updated on 2026-11-09
Numerical companion to the entry membership inference attack: it recomputes what the entry states and prints one line per check
Carries out the attack of the entry on recorded weather and shows what it reveals. A volunteer observer at Krems an der Donau reported the minimum and the maximum temperature of October 2024 for the days they were there, 7-13 and 20-26 October. A company learned a hypothesis from those reports that predicts the day's maximum temperature from its minimum, using the first d cosine functions of the rescaled minimum temperature as features. With d equal to the number of reported days, least squares reproduces every reported day exactly.
Run it without installing anything:uv run https://dictionaryofml.org/terms/membershipinferenceattack.py
uv downloads this script and the pinned NumPy and Matplotlib it needs, then runs it; the script fetches any input file it uses. To keep the output files, download membershipinferenceattack.py into a folder and run uv run membershipinferenceattack.py there. With NumPy and Matplotlib already installed, python3 membershipinferenceattack.py, from any directory — it writes its output files into the current directory. Fixed seeds, so the printed numbers reproduce exactly. Download membershipinferenceattack.py · Notebook · Open in Colab
Before running a downloaded copy, scan it: uvx bandit -q -t B102,B307,B602,B605 membershipinferenceattack.py (B102,B307,B602,B605 are the exec, eval and shell checks of bandit) reports nothing: the script runs no shell command and evaluates no generated code.
One cell per block of the script: the code, and what that code printed when it last ran here
"""membershipinferenceattack.py — numerical companion to the glossary entry
'membership inference attack'.
Purpose
-------
Carries out the attack of the entry on recorded weather and shows what it
reveals. A volunteer observer at Krems an der Donau reported the minimum
and the maximum temperature of October 2024 for the days they were there,
7-13 and 20-26 October. A company learned a hypothesis from those reports
that predicts the day's maximum temperature from its minimum, using the
first d cosine functions of the rescaled minimum temperature as features.
With d equal to the number of reported days, least squares reproduces every
reported day exactly.
An adversary who can query the hypothesis, and who reads the public record
of the GeoSphere Austria weather station at Krems (station 3805), computes
for every day of the month the squared error between the station's maximum
temperature and the hypothesis' prediction from the station's minimum. On
the reported days that loss is zero and on every other day it is not, so
the small losses name the training set, and because the reports come from
one person they name the days that person was at Krems.
The demo checks the entry's claims: the learned hypothesis interpolates the
reported days, which needs their minimum temperatures to be distinct; the
loss separates reported from unreported days with no overlap, so one
threshold classifies all 31 days correctly; the attack gets better as the
hypothesis is given more features; and adding a penalty term to the
training error closes the gap between the loss on the reported days and
the loss elsewhere, which costs the adversary almost all of its advantage
while lowering the loss outside the training set.
The attack is measured by its advantage: the largest value, over all
thresholds, of the fraction of the 14 reported days whose loss falls below
the threshold minus the fraction of the 17 other days of the month whose
loss does. An advantage of zero is a useless attack and one is a perfect
one. The threshold is chosen with the answer already known, so the
advantage is the most any threshold could achieve.
Deterministic: the archive is historical, so the readings are fixed. Needs
the network for the GeoSphere archive. Self-contained: numpy + matplotlib.
Blocks
------
[B-data] The 31 days of October 2024 at Krems: minimum and maximum
temperature per day, and the two stays, 7-13 and 20-26, that
the observer reported. Check that no reading is missing and
that the reported days carry distinct minimum temperatures,
which is what lets the hypothesis interpolate them.
[B-attack] The hypothesis with d = 14 cosine features learned from the 14
reported days, and the loss it incurs on every day of the
month. Check that the reported days incur no loss, that every
other day incurs more, and that one threshold therefore labels
all 31 days correctly.
[B-flex] The same attack for d = 2, ..., 14. Check that the attack's
advantage grows with the gap between the average loss on the
reported days and the average loss elsewhere, and that it is
small when the gap is.
[B-defend] The same d = 14 features, learned with the penalty term
lam * ||w||^2 added to the training error, the constant
feature left unpenalized. Check that the gap all but
vanishes, that the adversary keeps little advantage, and
that the loss outside the reported days falls at the same
time, so the penalty protects the reports and improves the
predictions elsewhere.
[B-plot] Write the per-day losses of both hypotheses and the per-d
table, plus the preview.
Outputs
-------
membershipinferenceattack_reported.csv : day, loss -- the reported days
membershipinferenceattack_unreported.csv : day, loss -- the other days
membershipinferenceattack_reportedreg.csv : the same, with the penalty
membershipinferenceattack_unreportedreg.csv : the same, with the penalty
membershipinferenceattack_gap.csv : d, trainerr, valerr, gap,
advantage
membershipinferenceattack.png : matplotlib preview (checking
only)
"""
# /// script
# requires-python = ">=3.10"
# dependencies = ["matplotlib==3.10.7", "numpy==2.3.4"]
# ///
import json
import urllib.request
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from pathlib import Path
OUT_DIR = Path(__file__).parent
ARCHIVE = "https://dataset.api.hub.geosphere.at/v1/station/historical/"
STATION = 3805 # Krems an der Donau
START, END = "2024-10-01", "2024-10-31"
STAYS = ((7, 13), (20, 26)) # the days the observer was there
REGPARAM = 10.0 # the penalty of the defended hypothesis
report = []
def check(name, ok):
report.append((name, bool(ok)))
print(f" [{'ok' if ok else 'FAIL'}] {name}")
31 days of October 2024 at station 3805; the observer reported 14 of them: 7-13, 20-26
The 31 days of October 2024 at Krems: minimum and maximum temperature per day, and the two stays, 7-13 and 20-26, that the observer reported. Check that no reading is missing and that the reported days carry distinct minimum temperatures, which is what lets the hypothesis interpolate them.
def fetch(parameters, start, end):
"""Daily parameters of one station from the GeoSphere Austria archive."""
url = (f"{ARCHIVE}klima-v2-1d?parameters={','.join(parameters)}"
f"&station_ids={STATION}&start={start}T00:00&end={end}T00:00")
with urllib.request.urlopen(url, timeout=180) as resp:
payload = json.load(resp)
values = payload["features"][0]["properties"]["parameters"]
return [np.array(values[p]["data"], dtype=float) for p in parameters]
tmin, tmax = fetch(["tlmin", "tlmax"], START, END)
day = np.arange(1, len(tmin) + 1)
reported = np.zeros(len(day), dtype=bool)
for first, last in STAYS:
reported[(day >= first) & (day <= last)] = True
m = int(reported.sum())
print(f" {len(day)} days of October 2024 at station {STATION}; the observer "
f"reported {m} of them: "
+ ", ".join(f"{a}-{b}" for a, b in STAYS))
check("[B-data] every day of the month carries both readings",
not np.isnan(tmin).any() and not np.isnan(tmax).any())
check("[B-data] the reported days carry distinct minimum temperatures, so "
"the design matrix of the reported days is invertible",
len(set(tmin[reported])) == m)
[ok] [B-data] every day of the month carries both readings [ok] [B-data] the reported days carry distinct minimum temperatures, so the design matrix of the reported days is invertible d = 14: largest loss on a reported day 3.2e-28, smallest loss elsewhere 0.23, median elsewhere 2.79, largest 47.7
The hypothesis with d = 14 cosine features learned from the 14 reported days, and the loss it incurs on every day of the month. Check that the reported days incur no loss, that every other day incurs more, and that one threshold therefore labels all 31 days correctly.
LO, HI = float(tmin.min()), float(tmin.max())
def features(xs, d):
"""The first d cosine features of the rescaled minimum temperature."""
scaled = (xs - LO) / (HI - LO) # into [0, 1]
return np.cos(np.outer(scaled, np.arange(d)) * np.pi)
def learn(d):
"""Least-squares weights from the reported days alone."""
return np.linalg.lstsq(features(tmin[reported], d), tmax[reported],
rcond=None)[0]
def losses(d):
"""Squared error of the prediction on every day of the month."""
weights = learn(d)
return (tmax - features(tmin, d) @ weights) ** 2
def advantage(loss):
"""The adversary declares a day reported when its loss falls below a
threshold. Of the reported days, the fraction with a loss at most the
threshold is the detection rate; of the days not reported, the fraction
with a loss at most the threshold is the false-alarm rate. The
advantage is the largest value of the detection rate minus the
false-alarm rate over all thresholds: zero whenever the two rates
agree, which is what answering the same way on every day gives, and one
when some threshold separates the two groups. The threshold is chosen
with the answer already known, so this is the most any threshold could
achieve."""
best, best_tau = 0.0, 0.0
for tau in np.unique(np.r_[loss, 0.0]):
value = float((loss[reported] <= tau).mean()
- (loss[~reported] <= tau).mean())
if value > best:
best, best_tau = value, float(tau)
return best, best_tau
loss_full = losses(m)
adv_full, tau_full = advantage(loss_full)
print(f" d = {m}: largest loss on a reported day "
f"{loss_full[reported].max():.1e}, smallest loss elsewhere "
f"{loss_full[~reported].min():.2f}, median elsewhere "
f"{np.median(loss_full[~reported]):.2f}, largest "
f"{loss_full[~reported].max():.1f}")
check(f"[B-attack] the reported days incur no loss and every other day does, "
f"so one threshold separates the two groups and the adversary's "
f"advantage is {adv_full:.2f}",
loss_full[reported].max() < 1e-6
and loss_full[reported].max() < loss_full[~reported].min()
and adv_full == 1.0)
check("[B-attack] the two stays are exactly the days of no loss, so the "
"small losses name when the observer was at Krems",
set(day[loss_full <= tau_full]) == set(day[reported]))
[ok] [B-attack] the reported days incur no loss and every other day does, so one threshold separates the two groups and the adversary's advantage is 1.00 [ok] [B-attack] the two stays are exactly the days of no loss, so the small losses name when the observer was at Krems d = 2: gap -0.46, advantage 0.10; d = 14: gap 14.29, advantage 1.00
The same attack for d = 2, ..., 14. Check that the attack's advantage grows with the gap between the average loss on the reported days and the average loss elsewhere, and that it is small when the gap is.
rows = []
for d in range(2, m + 1):
loss = losses(d)
adv, _ = advantage(loss)
tr, va = float(loss[reported].mean()), float(loss[~reported].mean())
rows.append((d, tr, va, va - tr, adv))
gaps = np.array([r[3] for r in rows])
advs = np.array([r[4] for r in rows])
agree = float(np.corrcoef(np.argsort(np.argsort(gaps)),
np.argsort(np.argsort(advs)))[0, 1])
print(f" d = 2: gap {rows[0][3]:.2f}, advantage {rows[0][4]:.2f}; "
f"d = {m}: gap {rows[-1][3]:.2f}, advantage {rows[-1][4]:.2f}")
check(f"[B-flex] the attack's advantage grows with the gap (rank agreement "
f"{agree:.2f}) and is perfect once the hypothesis interpolates",
agree > 0.8 and rows[-1][4] == 1.0 and rows[0][4] < 0.2)
[ok] [B-flex] the attack's advantage grows with the gap (rank agreement 0.90) and is perfect once the hypothesis interpolates penalty 10: average loss on the reported days 5.24, elsewhere 5.26, gap 0.02, advantage 0.15
The same d = 14 features, learned with the penalty term lam * ||w||^2 added to the training error, the constant feature left unpenalized. Check that the gap all but vanishes, that the adversary keeps little advantage, and that the loss outside the reported days falls at the same time, so the penalty protects the reports and improves the predictions elsewhere.
def learn_penalized(d, regparam):
"""Least squares on the reported days with the penalty term
regparam * ||w||^2 added, the constant feature left unpenalized."""
design = features(tmin[reported], d)
penalty = np.eye(d)
penalty[0, 0] = 0.0
return np.linalg.solve(design.T @ design / m + regparam * penalty,
design.T @ tmax[reported] / m)
weights_reg = learn_penalized(m, REGPARAM)
loss_reg = (tmax - features(tmin, m) @ weights_reg) ** 2
adv_reg, _ = advantage(loss_reg)
tr_reg, va_reg = float(loss_reg[reported].mean()), float(loss_reg[~reported].mean())
print(f" penalty {REGPARAM:g}: average loss on the reported days "
f"{tr_reg:.2f}, elsewhere {va_reg:.2f}, gap {va_reg - tr_reg:.2f}, "
f"advantage {adv_reg:.2f}")
check(f"[B-defend] the penalty closes the gap, from "
f"{rows[-1][3]:.2f} to {va_reg - tr_reg:.2f}, and leaves the adversary "
f"an advantage of {adv_reg:.2f} in place of {rows[-1][4]:.2f}",
abs(va_reg - tr_reg) < 0.5 and adv_reg < 0.25)
check(f"[B-defend] the loss outside the reported days falls at the same "
f"time, from {rows[-1][2]:.2f} to {va_reg:.2f}, so the penalty costs "
f"no accuracy there",
va_reg < rows[-1][2])
[ok] [B-defend] the penalty closes the gap, from 14.29 to 0.02, and leaves the adversary an advantage of 0.15 in place of 1.00 [ok] [B-defend] the loss outside the reported days falls at the same time, from 14.29 to 5.26, so the penalty costs no accuracy there 7/7 checks pass wrote the four CSV files and /Users/junga1/dictionaryappliedml/pythondemos/membershipinferenceattack.png
Write the per-day losses of both hypotheses and the per-d table, plus the preview.
with open(OUT_DIR / "membershipinferenceattack_reported.csv", "w") as fh:
fh.write("day,loss\n")
for d_, v in zip(day[reported], loss_full[reported]):
fh.write(f"{int(d_)},{v:.6e}\n")
with open(OUT_DIR / "membershipinferenceattack_unreported.csv", "w") as fh:
fh.write("day,loss\n")
for d_, v in zip(day[~reported], loss_full[~reported]):
fh.write(f"{int(d_)},{v:.4f}\n")
with open(OUT_DIR / "membershipinferenceattack_gap.csv", "w") as fh:
fh.write("d,trainerr,valerr,gap,advantage\n")
for d_, tr, va, gap, adv in rows:
fh.write(f"{d_},{tr:.5f},{va:.5f},{gap:.5f},{adv:.4f}\n")
for name, mask, loss_of, fmt in (
("membershipinferenceattack_reportedreg.csv", reported, loss_reg, "%.4f"),
("membershipinferenceattack_unreportedreg.csv", ~reported, loss_reg, "%.4f")):
with open(OUT_DIR / name, "w") as fh:
fh.write("day,loss\n")
for d_, v in zip(day[mask], loss_of[mask]):
fh.write(f"{int(d_)}," + (fmt % v) + "\n")
def stem_panel(ax, loss, title):
ax.stem(day[~reported], loss[~reported], linefmt="k-", markerfmt="ko",
basefmt=" ", label="day not reported")
marks = ax.stem(day[reported], loss[reported], linefmt="-",
markerfmt="s", basefmt=" ",
label="reported day (training set)")
plt.setp(marks.markerline, color="0.45", markersize=5)
plt.setp(marks.stemlines, color="0.45", linestyle="--")
for first, last in STAYS:
ax.annotate("", xy=(first, 44), xytext=(last, 44),
arrowprops=dict(arrowstyle="<->", lw=0.8))
ax.text((first + last) / 2, 45, "observer at Krems", ha="center",
fontsize=7)
ax.set_xlabel("day of October 2024")
ax.set_ylabel("loss of the prediction ($^\\circ$C$^2$)")
ax.set_title(title)
ax.set_ylim(0, 52)
ax.legend(frameon=False, fontsize=7, loc="upper center",
bbox_to_anchor=(0.5, -0.17))
fig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(14.4, 4.4))
stem_panel(ax1, loss_full, f"no penalty ($d = {m}$): the loss names the days")
ax2.plot([r[0] for r in rows], [r[4] for r in rows], "ko-")
ax2.set_xlabel("number $d$ of cosine features of the hypothesis")
ax2.set_ylabel("advantage of the attack")
ax2.set_title("more features, better attack")
ax2.set_ylim(0, 1.08)
stem_panel(ax3, loss_reg,
f"penalty $\\lambda = {REGPARAM:g}$: the loss names nothing")
fig.tight_layout()
fig.savefig(OUT_DIR / "membershipinferenceattack.png", dpi=110)
n_ok = sum(ok for _, ok in report)
print(f"\n{n_ok}/{len(report)} checks pass")
print(f"wrote the four CSV files and "
f"{OUT_DIR / 'membershipinferenceattack.png'}")
if n_ok != len(report):
raise SystemExit(1)

B-plot writes when the script runs