\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath,amssymb}
\usepackage{algorithm,algpseudocode}
\usepackage{bm}
\usepackage{hyperref}
\usepackage{float}
\usepackage{subcaption}
\usepackage{enumitem}
\usepackage{xcolor}
\definecolor{darkgreen}{rgb}{0.0, 0.5, 0.0}
\definecolor{aaltoBlue}{RGB}{0, 51, 102}
\definecolor{pinegreen}{cmyk}{0.92,0,0.59,0.25}
\definecolor{royalblue}{cmyk}{1,0.50,0,0}
\definecolor{lavander}{cmyk}{0,0.48,0,0}
\definecolor{violet}{cmyk}{0.79,0.88,0,0}
\definecolor{lightblue}{RGB}{173, 216, 230}
\usepackage{tikz}
\usepackage{pgfplots}
\usetikzlibrary{calc,arrows.meta,positioning,fit,shapes.geometric,shapes.misc,intersections,patterns,decorations.pathreplacing,backgrounds,matrix}
\usepgfplotslibrary{fillbetween,groupplots}
\setcounter{secnumdepth}{-\maxdimen}
% Long unbreakable tokens (the ISO/IEC author names in citations, term names
% in the title) otherwise overflow the margin by a few points; a modest
% emergency stretch lets TeX loosen the paragraph instead.
\emergencystretch=2em
\usepackage{mleftright}
% Project macros used by this entry (auto-extracted).
\newcommand{\datapoint}{\vz}
\newcommand{\dataset}{\mathcal{D}}
\newcommand{\feature}{x}
\newcommand{\featuredim}{d}
\newcommand{\featureidx}{j}
\newcommand{\hypothesis}{h}
\newcommand{\learnthypothesis}{\widehat{\hypothesis}}
\newcommand{\lipschitzconstant}{L}
\newcommand{\normgeneric}[2]{\mleft\lVert #1 \mright\rVert_{#2}}
\newcommand{\pair}[2]{\left( #1,#2 \right)}
\newcommand{\regparam}{\alpha}
\newcommand{\samplesize}{m}
\newcommand{\trainset}{\dataset^{(\rm train)}}
\newcommand{\truelabel}{y}
\newcommand{\vw}[0]{{\bf w}}
\newcommand{\vz}[0]{{\bf z}}
\newcommand{\weights}{\vw}

\begin{document}

\section{membership inference attack}

\subsection*{Author and editors}
Alexander Jung\textsuperscript{*}, Department of Computer Science, Aalto University, Espoo, Finland (ORCID \mbox{\href{https://orcid.org/0000-0001-7538-0990}{0000-0001-7538-0990}}) \\
Editors: Konstantina Olioumtsevits (ORCID \mbox{\href{https://orcid.org/0000-0003-2697-8659}{0000-0003-2697-8659}}), Ekkehard Schnoor (ORCID \mbox{\href{https://orcid.org/0000-0001-8252-1499}{0000-0001-8252-1499}}), Aalto University, Espoo, Finland \\
\textsuperscript{*}Corresponding author: \href{mailto:alex.jung@aalto.fi}{alex.jung@aalto.fi}



\subsection{Abstract}
A membership inference attack aims at determining whether a
		particular data point was used to train a machine learning (ML)
		model. The basic idea is that a learned hypothesis
		delivers more accurate predictions for data points
		that are similar to those used for training. An adversary
		queries the learned hypothesis and compares the loss
		of its prediction against a threshold. Mitigation strategies
		include regularization, which makes an ML method
		generalize beyond the training set: the learned
		hypothesis then behaves similarly for data points in
		the training set and for those outside it, which makes it harder
		for an adversary to guess the data points of the
		training set.

\subsection{Definition}
A volunteer observer
		at Krems an der Donau reports the minimum and the maximum
		temperature for selected days. Each day is one data point
		$\datapoint = \pair{\feature}{\truelabel}$, whose feature
		$\feature$ is that day's minimum temperature and whose label
		$\truelabel$ is that day's maximum temperature, and the reported
		days form the training set $\trainset$. A company learns a
		hypothesis $\learnthypothesis$ from $\trainset$ to predict
		the maximum temperature of a day from its minimum. The company
		publishes
		$\learnthypothesis$, so anyone can query it at any minimum
		temperature. The GeoSphere weather station at Krems publishes the
		minimum and the maximum temperature of every day. Anyone with both
		can try to determine whether the report of a specific day was used
		for training. Since the reports come from one person, a yes
		tells that the observer was around Krems an der Donau on that day.
		Whether the observer was at Krems on a specific day is
		personal data, even though a temperature is not
		(European Parliament and Council of the European Union, 2016, Art.~4). This scenario is an example of a
		membership inference attack: a privacy attack where an
		adversary tries to determine whether a particular data point
		was used to train a machine learning (ML) model (Shokri et al., 2017;
		Vassilev et al., 2024, Sect.~2.4.2).
\par
		For a specific day, the adversary reads that day's $\feature$ and
		$\truelabel$ from the station's record, computes the
		prediction $\learnthypothesis(\feature)$, and compares it
		against $\truelabel$. A
		loss $\big(\truelabel - \learnthypothesis(\feature)\big)^{2}$
		below a threshold indicates that the report of that day was used
		for training.
\par
		Fig.~\ref{fig_membershipinferenceattack} carries this attack
		out on the 31 days of October 2024 at Krems. One threshold labels
		all 31 days correctly: the loss is zero on every day the
		observer reported and never below $0.23\,^{\circ}$C$^{2}$ on any
		other day. The days it declares reported form two blocks of
		consecutive days, 7 to 13 and 20 to 26 October, which is when the
		observer was there. The hypothesis behind the figure is
		obtained by linear regression on the features
		$\feature'_{\featureidx} = \cos\big((\featureidx-1) \pi u\big)$ for
		$\featureidx = 1, \ldots, \featuredim$, with $u$ the minimum
		temperature $\feature$ rescaled to $[0,1]$ over the month; for
		$\featuredim = 14$, one feature per reported day,
		least squares reproduces every reported day exactly.
\begin{figure}[H]
			\centering
			\begin{tikzpicture}
			\begin{axis}[
				width=0.92\linewidth, height=0.46\linewidth,
				xlabel={day of October 2024},
				ylabel={loss
					$\big(\truelabel - \learnthypothesis(\feature)\big)^{2}$
					($^{\circ}$C$^{2}$)},
				axis lines=left,
				xmin=0, xmax=32, ymin=0, ymax=58,
				xtick={1,5,10,15,20,25,30},
				legend style={draw=none, fill=none, font=\small,
					at={(0.5,-0.32)}, anchor=north, legend columns=-1,
					column sep=10pt},
				tick label style={font=\small}, label style={font=\small},
				]
				\addplot[ycomb, black, thick, mark=*, mark size=1.8pt]
					table[col sep=comma, x=day, y=loss]{membershipinferenceattack_unreported.csv};
				\addlegendentry{day not reported, $\datapoint \notin \trainset$}
				\addplot[ycomb, gray, thick, dashed, mark=square*, mark size=2.6pt]
					table[col sep=comma, x=day, y=loss]{membershipinferenceattack_reported.csv};
				\addlegendentry{reported day, $\datapoint \in \trainset$}
				\draw[<->, thin] (axis cs:7,47) -- (axis cs:13,47)
					node[midway, above, font=\small] {observer at Krems};
				\draw[<->, thin] (axis cs:20,47) -- (axis cs:26,47)
					node[midway, above, font=\small] {observer at Krems};
			\end{axis}
			\end{tikzpicture}
		\caption{The loss the published hypothesis incurs on
			each day of October 2024 at Krems an der Donau, computed by an
			adversary from the weather station's own record. Every day that
			the observer did not report carries a solid stem, drawn from
			zero to its loss and topped by a disc. The 14 days the
			observer did report carry no stem at all: the squares resting
			on the horizontal axis mark a loss of zero. The days
			those squares mark are 7 to 13 and 20 to 26 October, the two
			stays, so the losses say not only which
			days were used for training but also when
			the observer was at Krems. Data generated by
			\texttt{pythondemos/membershipinferenceattack.py}
			\label{fig_membershipinferenceattack}}
		\end{figure}
		The number $\featuredim$ sets the effective dimension of the underlying
		hypothesis space and so how far the ML method can overfit.
		Fig.~\ref{fig_membershipinferenceattack_flex} plots how much the
		attack gains from that. Fix a threshold $\tau$ on the
		loss. Of the 14
		days the observer reported, the fraction incurring a loss of
		at most $\tau$ is the detection rate. Of the 17 days of the month
		the observer did not report, the fraction incurring a loss of
		at most $\tau$ is the false-alarm rate. The advantage of the
		attack is the largest value of the detection rate minus the
		false-alarm rate over all thresholds. An adversary that answers yes
		on every day, or no on every day, leaves the two rates equal and
		the advantage zero, which is why the advantage measures what the
		attack gains over guessing, and a threshold that separates
		the two groups makes it one. The threshold is chosen with the answer already known, so the
		advantage is the most any threshold could achieve rather than what
		an adversary would obtain. With two features, it is $0.10$,
		and it reaches one once $\featuredim$ reaches the number of
		reported days.
\begin{figure}[H]
			\centering
			\begin{tikzpicture}
			\begin{axis}[
				width=0.78\linewidth, height=0.42\linewidth,
				xlabel={number $\featuredim$ of features of the
					hypothesis},
				ylabel={advantage of the attack},
				axis lines=left,
				xmin=1.5, xmax=14.5, ymin=0, ymax=1.12,
				xtick={2,4,6,8,10,12,14},
				tick label style={font=\small}, label style={font=\small},
				]
				\addplot[black, thick, mark=*, mark size=1.8pt]
					table[col sep=comma, x=d, y=advantage]{membershipinferenceattack_gap.csv};
			\end{axis}
			\end{tikzpicture}
		\caption{How the attack of
			Fig.~\ref{fig_membershipinferenceattack} improves as the
			hypothesis space grows. The curve is the advantage, the
			largest difference over all thresholds between the fraction of
			the 14 reported days whose loss falls below the threshold
			and the fraction of the 17 other days whose loss does,
			for a hypothesis learned from the same reported days with
			$\featuredim$ features. Data generated by
			\texttt{pythondemos/membershipinferenceattack.py}
			\label{fig_membershipinferenceattack_flex}}
		\end{figure}
		The attack succeeds only to the extent that the learned
		hypothesis incurs a smaller loss inside the
		training set than outside it, which is what the generalization gap
		measures. Under a probabilistic model for how the data points
		and their membership of the training set arise, the success of
		such an
		attack is tied to that gap. The quantity is the same
		difference as above, with the two rates replaced by
		probabilities under that model. For a loss bounded by a
		constant $B$, an adversary that answers yes with a
		probability decreasing in the loss has a difference
		that grows in proportion to the expected generalization gap
		(Yeom et al., 2018, Thm.~2). Regularization
		controls that gap: for a convex loss that is
		Lipschitz with constant $\lipschitzconstant$, adding the
		penalty term $\regparam \normgeneric{\weights}{2}^{2}$ bounds
		the expected generalization gap by
		$2\lipschitzconstant^{2}/(\regparam \samplesize)$
		(Shalev-Shwartz and Ben-David, 2014, Cor.~13.6).
		Fig.~\ref{fig_membershipinferenceattack_reg} repeats the
		attack against the same features learned with that
		penalty term, and the picture no longer tells the two groups
		apart. Regularization therefore serves two purposes,
		accurate predictions outside the training set
		(Hastie et al., 2009, Sect.~7.4) and a defence
		against a membership inference attack
		(Shokri et al., 2017, Sect.~VIII). For the observer at Krems, that is
		the difference between a published hypothesis that reveals
		nothing about them and one that reveals the days they were in town.
\begin{figure}[H]
			\centering
			\begin{tikzpicture}
			\begin{axis}[
				width=0.92\linewidth, height=0.46\linewidth,
				xlabel={day of October 2024},
				ylabel={loss
					$\big(\truelabel - \learnthypothesis(\feature)\big)^{2}$
					($^{\circ}$C$^{2}$)},
				axis lines=left,
				xmin=0, xmax=32, ymin=0, ymax=58,
				xtick={1,5,10,15,20,25,30},
				legend style={draw=none, fill=none, font=\small,
					at={(0.5,-0.32)}, anchor=north, legend columns=-1,
					column sep=10pt},
				tick label style={font=\small}, label style={font=\small},
				]
				\addplot[ycomb, black, thick, mark=*, mark size=1.8pt]
					table[col sep=comma, x=day, y=loss]{membershipinferenceattack_unreportedreg.csv};
				\addlegendentry{day not reported, $\datapoint \notin \trainset$}
				\addplot[ycomb, gray, thick, dashed, mark=square*, mark size=2.6pt]
					table[col sep=comma, x=day, y=loss]{membershipinferenceattack_reportedreg.csv};
				\addlegendentry{reported day, $\datapoint \in \trainset$}
				\draw[<->, thin] (axis cs:7,47) -- (axis cs:13,47)
					node[midway, above, font=\small] {observer at Krems};
				\draw[<->, thin] (axis cs:20,47) -- (axis cs:26,47)
					node[midway, above, font=\small] {observer at Krems};
			\end{axis}
			\end{tikzpicture}
		\caption{The same attack as in
			Fig.~\ref{fig_membershipinferenceattack}, on the same axes,
			against the hypothesis learned from the same 14 reported
			days with the penalty term
			$\regparam \normgeneric{\weights}{2}^{2}$ for
			$\regparam = 10$. The reported days now carry dashed stems
			topped by a square, of much the same height as the solid stems
			topped by a disc that mark the days the observer did not
			report, so no threshold picks them out.
			Their average loss rises from $0$ to
			$5.24\,^{\circ}$C$^{2}$ while the average loss elsewhere
			falls from $14.29$ to $5.26\,^{\circ}$C$^{2}$: the
			generalization gap closes to $0.02\,^{\circ}$C$^{2}$ and the
			adversary's advantage drops from one to at most $0.15$. Data generated
			by \texttt{pythondemos/membershipinferenceattack.py}
			\label{fig_membershipinferenceattack_reg}}
		\end{figure}

\subsection{Cross References}
attack, privacy attack, personal data, regularization, DP, overfitting, training error, validation error, training set

\subsection{References}
\begin{enumerate}
  \item European Parliament and Council of the European Union (2016). Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (Text with EEA relevance). Official Journal of the European Union, L~119/1, May 4. URL: https://eur-lex.europa.eu/eli/reg/2016/679/oj.
  \item Yeom et al. (2018). Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting. In: 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pp.~268--282.
  \item Shalev-Shwartz and Ben-David (2014). Understanding Machine Learning: From Theory to Algorithms. Cambridge Univ. Press, New York, NY, USA.
  \item Shokri et al. (2017). Membership Inference Attacks Against Machine Learning Models. In: 2017 IEEE Symp. Secur. Privacy, pp.~3--18.
  \item Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed., Springer Science+Business Media, New York, NY, USA.
  \item Vassilev et al. (2024). Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, Gaithersburg, MD.
\end{enumerate}

\end{document}