{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3",
   "language": "python"
  },
  "language_info": {
   "name": "python"
  },
  "colab": {
   "name": "accuracy.ipynb"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# accuracy \u2014 Python demo\n\nNumerical companion to the entry [accuracy](https://dictionaryofml.org/terms/accuracy.html) of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org/): it recomputes what the entry states and prints one line per check.\n\nIllustrate the concept of accuracy: the fraction of data points whose predicted label matches the true label. A simple linear classifier is applied to synthetic 2D Gaussian data, and each data point is marked as correctly or incorrectly classified.\n\nRequires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Generated from [`pythondemos/accuracy.py`](https://dictionaryofml.org/terms/accuracy.py); CC BY 4.0."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Notebook shim: the script resolves output paths relative to __file__,\n# which a notebook kernel does not define; everything lands in the\n# working directory instead.\nimport os\n__file__ = os.path.join(os.getcwd(), \"accuracy.py\")\nos.makedirs(\"pythondemos\", exist_ok=True)"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "\"\"\"\naccuracy.py \u2014 Accuracy of a linear classifier on a 2D binary dataset.\n\nPurpose\n-------\nIllustrate the concept of accuracy: the fraction of data points\nwhose predicted label matches the true label.  A simple linear\nclassifier is applied to synthetic 2D Gaussian data, and each\ndata point is marked as correctly or incorrectly classified.\n\nData generation\n---------------\nTwo classes (y=0 and y=1) are generated from isotropic Gaussian\ndistributions with different means:\n    class 0: x ~ N(mu0, sigma^2 I),  mu0 = (1, 2)\n    class 1: x ~ N(mu1, sigma^2 I),  mu1 = (3, 1)\n    sigma = 0.8, 30 points per class\n\nClassifier\n----------\nA linear decision boundary is constructed using the direction\nbetween the two class means (Fisher's discriminant direction):\n    w = (mu1 - mu0) / ||mu1 - mu0||\n    b = -w^T * midpoint,  where midpoint = (mu0 + mu1) / 2\nA data point x is classified as y=1 if w^T x + b > 0, else y=0.\n\nThis is not a trained classifier \u2014 it uses the known class means\ndirectly.  The purpose is to produce a clean, reproducible example\nwith a few misclassified points near the decision boundary.\n\nOutput\n------\n  pythondemos/accuracy_data.csv     \u2014 columns: x1, x2, label, predicted, correct\n  pythondemos/accuracy_boundary.csv \u2014 columns: x1, x2 (two endpoints of boundary)\n  pythondemos/accuracy.png          \u2014 matplotlib preview\n\"\"\"\n\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom pathlib import Path\n\nOUT_DIR = Path(__file__).parent\n\n# Fix random seed for reproducibility (re-running produces identical CSVs)\nnp.random.seed(42)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[S-generate-two-overlapping]**"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# 30 data points per class, 60 total.  The centers are chosen so\n# that the two clouds overlap slightly, producing a few\n# misclassifications.\nn_per_class = 30\nmu0 = np.array([1.0, 2.0])   # center of class 0\nmu1 = np.array([3.0, 1.0])   # center of class 1\nsigma = 0.8                   # spread (same for both classes)\n\nX0 = mu0 + sigma * np.random.randn(n_per_class, 2)  # class 0 data points\nX1 = mu1 + sigma * np.random.randn(n_per_class, 2)  # class 1 data points\nX = np.vstack([X0, X1])                              # (60, 2) feature vectors\ny = np.array([0] * n_per_class + [1] * n_per_class)  # true labels"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[S-linear-classifier-h]**"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# The weight vector w points from mu0 toward mu1 (the direction\n# that best separates the two class centers).  The bias b is\n# chosen so that the decision boundary passes through the\n# midpoint of the two centers, the best possible boundary for\n# two equally spread, equally sized classes.\nw = mu1 - mu0                   # direction between class means\nw = w / np.linalg.norm(w)       # normalize to unit length\nmidpoint = 0.5 * (mu0 + mu1)   # midpoint between the means\nb = -w @ midpoint               # bias: boundary passes through midpoint\n\n# Compute the classifier score for each data point.\n# Positive score \u2192 predict class 1; negative score \u2192 predict class 0.\nscores = X @ w + b\ny_pred = (scores > 0).astype(int)\n\n# Determine which predictions are correct (1) or incorrect (0)\ncorrect = (y_pred == y).astype(int)\naccuracy = correct.mean()\n\nprint(f\"Accuracy: {accuracy:.2f} ({correct.sum()}/{len(y)})\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[S-verify]**"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "assert 0.8 < accuracy < 1.0, f\"Accuracy out of expected range: {accuracy}\"\nassert len(y) == 2 * n_per_class, f\"Unexpected sample size: {len(y)}\""
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[S-save-csv]**"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# accuracy_data.csv: one row per data point, columns:\n#   x1, x2    \u2014 feature values\n#   label     \u2014 true class (0 or 1)\n#   predicted \u2014 predicted class (0 or 1)\n#   correct   \u2014 1 if prediction matches label, 0 otherwise\ndata = np.column_stack([X, y, y_pred, correct])\nheader = \"x1,x2,label,predicted,correct\"\nnp.savetxt(OUT_DIR / \"accuracy_data.csv\", data,\n           delimiter=\",\", header=header, comments=\"\",\n           fmt=[\"%.6f\", \"%.6f\", \"%d\", \"%d\", \"%d\"])\n\n# accuracy_boundary.csv: two endpoints of the decision boundary line.\n# The boundary is the set {x : w^T x + b = 0}, which is a line\n# in 2D.  Solving for x2: x2 = -(w1*x1 + b) / w2.\nx1_range = np.array([X[:, 0].min() - 0.5, X[:, 0].max() + 0.5])\nx2_boundary = -(w[0] * x1_range + b) / w[1]\nnp.savetxt(OUT_DIR / \"accuracy_boundary.csv\",\n           np.column_stack([x1_range, x2_boundary]),\n           delimiter=\",\", header=\"x1,x2\", comments=\"\")\n\nprint(f\"Saved {OUT_DIR / 'accuracy_data.csv'}\")\nprint(f\"Saved {OUT_DIR / 'accuracy_boundary.csv'}\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[S-nonlinear-overfit-classifier]**"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# RBF-weighted score:\n#   f(x) = sum_{class 1} exp(-||x-x_i||^2 / (2 sigma_rbf^2))\n#        - sum_{class 0} exp(-||x-x_i||^2 / (2 sigma_rbf^2))\n# A small bandwidth sigma_rbf makes the classifier approach\n# 1-NN, which achieves perfect training accuracy but wiggles\n# sharply near misclassified points.  This illustrates how\n# optimizing accuracy alone can yield a highly non-smooth\n# decision boundary that is sensitive to feature perturbations.\nsigma_rbf = 0.10\n\ndef rbf_score(points, centers, s):\n    d2 = ((points[:, None, :] - centers[None, :, :]) ** 2).sum(-1)\n    return np.exp(-d2 / (2.0 * s ** 2)).sum(axis=1)\n\n# Grid for extracting the 0-level contour.\nx1_min, x1_max = X[:, 0].min() - 0.6, X[:, 0].max() + 0.6\nx2_min, x2_max = X[:, 1].min() - 0.6, X[:, 1].max() + 0.6\nxx, yy = np.meshgrid(\n    np.linspace(x1_min, x1_max, 500),\n    np.linspace(x2_min, x2_max, 500),\n)\ngrid = np.stack([xx.ravel(), yy.ravel()], axis=1)\nZ = rbf_score(grid, X1, sigma_rbf) - rbf_score(grid, X0, sigma_rbf)\nZ = Z.reshape(xx.shape)\n\n# Verify perfect training accuracy of the RBF classifier.\ntrain_scores = rbf_score(X, X1, sigma_rbf) - rbf_score(X, X0, sigma_rbf)\nrbf_pred = (train_scores > 0).astype(int)\nrbf_acc = (rbf_pred == y).mean()\nassert rbf_acc == 1.0, f\"RBF classifier not at perfect accuracy: {rbf_acc}\"\n\n# Extract the 0-level contour (may consist of several pieces).\n# The axes exists only for this extraction and is closed unsaved;\n# the labels and title keep it within the figure rules anyway.\nfig_tmp, ax_tmp = plt.subplots()\nax_tmp.set_xlabel(\"feature x1\")\nax_tmp.set_ylabel(\"feature x2\")\nax_tmp.set_title(\"contour extraction (not saved)\")\ncs = ax_tmp.contour(xx, yy, Z, levels=[0])\nsegs = cs.allsegs[0]\nplt.close(fig_tmp)\n\n# Write a single CSV with segments separated by \"nan,nan\" rows.\n# pgfplots with [unbounded coords=jump] starts a new path at each\n# NaN coordinate, so the resulting plot has disjoint pieces.\nwith open(OUT_DIR / \"accuracy_overfit_boundary.csv\", \"w\") as f:\n    f.write(\"x1,x2\\n\")\n    for i, seg in enumerate(segs):\n        if i > 0:\n            f.write(\"nan,nan\\n\")\n        for px, py in seg:\n            f.write(f\"{px:.6f},{py:.6f}\\n\")\nprint(f\"Saved {OUT_DIR / 'accuracy_overfit_boundary.csv'}\"\n      f\"  ({len(segs)} contour segment(s))\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[S-preview-plot]**"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Blue circles: class 0, correctly classified\n# Red squares:  class 1, correctly classified\n# Black open circles: misclassified (either class)\n# Dashed line: decision boundary\nfig, ax = plt.subplots(figsize=(4.5, 3.5))\n\nmask_c = correct == 1  # correctly classified mask\nax.scatter(X[mask_c & (y == 0), 0], X[mask_c & (y == 0), 1],\n           c=\"tab:blue\", marker=\"o\", edgecolors=\"k\", linewidths=0.3,\n           s=30, label=r\"$y=0$, correct\", zorder=3)\nax.scatter(X[mask_c & (y == 1), 0], X[mask_c & (y == 1), 1],\n           c=\"tab:red\", marker=\"s\", edgecolors=\"k\", linewidths=0.3,\n           s=30, label=r\"$y=1$, correct\", zorder=3)\n\nmask_w = correct == 0  # incorrectly classified mask\nax.scatter(X[mask_w, 0], X[mask_w, 1],\n           c=\"none\", marker=\"o\", edgecolors=\"black\", linewidths=1.5,\n           s=80, label=\"incorrect\", zorder=4)\n\nax.plot(x1_range, x2_boundary, \"k--\", linewidth=1.0, label=\"linear $h$\")\nfor i, seg in enumerate(segs):\n    ax.plot(seg[:, 0], seg[:, 1], color=\"tab:green\", linewidth=1.2,\n            label=\"overfit $h'$\" if i == 0 else None)\n\nax.set_xlabel(r\"$x_1$\")\nax.set_ylabel(r\"$x_2$\")\nax.set_title(f\"accuracy = {accuracy:.2f}\")\nax.legend(fontsize=7, loc=\"upper left\", frameon=False)\nax.tick_params(labelsize=8)\n\nfig.tight_layout()\nout = OUT_DIR / \"accuracy.png\"\nfig.savefig(out, bbox_inches=\"tight\", dpi=110)\nprint(f\"Saved {out}\")\nplt.close()"
  }
 ]
}