{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3",
   "language": "python"
  },
  "language_info": {
   "name": "python"
  },
  "colab": {
   "name": "zerooneloss.ipynb"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# 0/1 loss \u2014 Python demo\n\nNumerical companion to the entry [0/1 loss](https://dictionaryofml.org/terms/zerooneloss.html) of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org/): it recomputes what the entry states and prints one line per check.\n\nNumerical companion to the glossary entry 'zerooneloss'. Each of the 365 days of 2024 at Krems (GeoSphere station 3805) is a data point: its feature x is the lowest temperature of the day, its label y is +1 if the lowest temperature of the following day is below zero (frost) and -1 otherwise. A frost warning compares today's lowest temperature with a threshold tau: the real-valued hypothesis h(x) = tau - x predicts frost when it is positive, and the thresholds form a one-parameter family of classifiers.\n\nRequires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Generated from [`pythondemos/zerooneloss.py`](https://dictionaryofml.org/terms/zerooneloss.py); CC BY 4.0.\n\nBefore running a downloaded copy, scan it: `uvx bandit -q -t B102,B307,B602,B605 zerooneloss.py` (B102,B307,B602,B605 are the exec, eval and shell checks of [bandit](https://bandit.readthedocs.io/)) reports nothing: the script runs no shell command and evaluates no generated code."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Notebook shim: the script resolves output paths relative to __file__,\n# which a notebook kernel does not define; everything lands in the\n# working directory instead.\nimport os\n__file__ = os.path.join(os.getcwd(), \"zerooneloss.py\")\nos.makedirs(\"pythondemos\", exist_ok=True)"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "\"\"\"The 0/1 loss of a frost warning at Krems an der Donau: a staircase in\nthe threshold against the smooth curve of the logistic loss.\n\nPurpose\n-------\nNumerical companion to the glossary entry 'zerooneloss'.  Each of the\n365 days of 2024 at Krems (GeoSphere station 3805) is a data point: its\nfeature x is the lowest temperature of the day, its label y is +1 if the\nlowest temperature of the following day is below zero (frost) and -1\notherwise.  A frost warning compares today's lowest temperature with a\nthreshold tau: the real-valued hypothesis h(x) = tau - x predicts frost\nwhen it is positive, and the thresholds form a one-parameter family of\nclassifiers.\n\nThe demo checks the entry's claims: the 0/1 loss of one day is a step\nin tau, while its logistic loss (divided by ln 2) and its hinge loss\nare convex curves that lie on or above the step and equal 1 where the\nstep jumps; the average 0/1 loss over the year is piecewise constant in\ntau, equal to one minus the accuracy, with a flat set of minimizers and\na zero slope wherever the slope exists; the average logistic loss and\nthe average hinge loss are convex in tau, lie above the staircase and\nhave one minimizer each, at which the staircase is close to its\nminimum; and the 0/1 loss does not change when the hypothesis is\nmultiplied by a positive factor, while the logistic loss does.\n\nDeterministic: no random numbers.  Self-contained: numpy + matplotlib\nonly; the GeoSphere download falls back to the committed CSV.\n\nBlocks\n------\n[B-data]    Download the daily lowest temperature of 2024 at Krems or\n            read the committed copy; 366 days give 365 (feature, label)\n            pairs, and the frost days are counted.\n[B-oneday]  The three losses of one day, the first day of autumn that was\n            followed by frost, as functions of the threshold tau: the 0/1\n            loss jumps once, at tau equal to the day's temperature; the\n            logistic loss / ln 2 and the hinge loss decrease, equal 1 at\n            the jump and never fall below the 0/1 loss.\n[B-stair]   The average 0/1 loss over the 365 days is a staircase in tau:\n            constant between consecutive distinct temperatures, one minus\n            the accuracy, with the never-frost and always-frost baselines\n            at its two ends, a flat interval of minimizers, and a\n            difference quotient of exactly zero away from the jumps.\n[B-smooth]  The average logistic loss / ln 2 and the average hinge loss\n            are convex in tau and lie above the staircase; each has one\n            minimizer, and the average 0/1 loss there is compared with\n            the smallest average 0/1 loss.\n[B-scale]   Multiplying the hypothesis by a positive factor leaves the\n            average 0/1 loss unchanged and changes the average logistic\n            loss.\n[B-plot]    CSVs for the figure and the matplotlib preview.\n\nOutputs\n-------\nzerooneloss_weather.csv : date, tmin (committed copy of the download).\nzerooneloss_oneday.csv  : tau, zo, log2, hinge for the one day.\nzerooneloss_stair.csv   : tau, zo: the staircase of the average 0/1 loss\n                          (one row per step, for a const plot).\nzerooneloss_curves.csv  : tau, log2, hinge: the two average surrogate\n                          losses on a grid.\nzerooneloss.png         : matplotlib preview (checking only).\n\"\"\"\n\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"matplotlib==3.10.7\", \"numpy==2.3.4\"]\n# ///\n\nimport json\nimport urllib.request\nfrom pathlib import Path\n\nimport numpy as np\nimport matplotlib\n\nmatplotlib.use(\"Agg\")\nimport matplotlib.pyplot as plt\n\nOUT_DIR = Path(__file__).parent\nLN2 = float(np.log(2.0))\n\nreport = []\n\n\ndef check(name, ok):\n    report.append((name, bool(ok)))\n    print(f\"  [{'ok' if ok else 'FAIL'}] {name}\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-data]** Download the daily lowest temperature of 2024 at Krems or read the committed copy; 366 days give 365 (feature, label) pairs, and the frost days are counted."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"[B-data] daily lowest temperature at Krems, 2024\")\nURL = (\"https://dataset.api.hub.geosphere.at/v1/station/historical/\"\n       \"klima-v2-1d?parameters=tlmin&station_ids=3805\"\n       \"&start=2024-01-01&end=2024-12-31\")\nCSV = OUT_DIR / \"zerooneloss_weather.csv\"\ntry:\n    with urllib.request.urlopen(URL, timeout=120) as resp:\n        payload = json.load(resp)\n    params = payload[\"features\"][0][\"properties\"][\"parameters\"]\n    records = [(stamp[:10], lo) for stamp, lo in\n               zip(payload[\"timestamps\"], params[\"tlmin\"][\"data\"])]\n    if any(lo is None for _, lo in records):\n        raise OSError(\"a day is missing its temperature\")\n    with open(CSV, \"w\") as fh:\n        fh.write(\"date,tmin\\n\")\n        for day, lo in records:\n            fh.write(f\"{day},{lo}\\n\")\n    print(f\"    downloaded {len(records)} days\")\nexcept OSError as err:\n    print(f\"    download failed ({err}); reading the committed CSV\")\n    rows = CSV.read_text().strip().splitlines()[1:]\n    records = [(d, float(a)) for d, a in (r.split(\",\") for r in rows)]\n\ncheck(\"[B-data] 366 days for 2024\", len(records) == 366)\ndates = [d for d, _ in records]\ntlmin = np.array([lo for _, lo in records], dtype=float)\nx = tlmin[:-1]                                  # today's lowest temperature\ny = np.where(tlmin[1:] < 0.0, 1.0, -1.0)        # +1: frost on the next day\nm = len(x)\nn_frost = int(np.sum(y > 0))\ncheck(f\"[B-data] m = {m} data points; frost follows {n_frost} of them \"\n      f\"(fraction {n_frost / m:.3f})\", m == 365 and 0 < n_frost < m)\n\n\n# ---- the three losses of one data point along the threshold family\ndef zo_loss(margin):\n    \"\"\"0/1 loss: 1 unless the margin is positive.\"\"\"\n    return np.where(margin > 0.0, 0.0, 1.0)\n\n\ndef log2_loss(margin):\n    \"\"\"Logistic loss divided by ln 2, so that it equals 1 at margin 0.\"\"\"\n    return np.logaddexp(0.0, -margin) / LN2\n\n\ndef hinge_loss(margin):\n    return np.maximum(0.0, 1.0 - margin)\n\n\ndef margins(tau, scale=1.0):\n    \"\"\"Margins y * h(x) of all days for the hypothesis h(x) = scale*(tau-x).\"\"\"\n    return y * scale * (tau - x)\n\n\ndef avg_zo(tau, scale=1.0):\n    return float(np.mean(zo_loss(margins(tau, scale))))\n\n\ndef avg_log2(tau, scale=1.0):\n    return float(np.mean(log2_loss(margins(tau, scale))))\n\n\ndef avg_hinge(tau, scale=1.0):\n    return float(np.mean(hinge_loss(margins(tau, scale))))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-oneday]** The three losses of one day, the first day of autumn that was followed by frost, as functions of the threshold tau: the 0/1 loss jumps once, at tau equal to the day's temperature; the logistic loss / ln 2 and the hinge loss decrease, equal 1 at the jump and never fall below the 0/1 loss."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"[B-oneday] the losses of one day as functions of the threshold\")\nautumn = [i for i in range(m) if dates[i] >= \"2024-09-01\" and y[i] > 0]\ni_day = autumn[0]\nx_day, y_day = float(x[i_day]), float(y[i_day])\nday_label = dates[i_day][8:10] + \"/\" + dates[i_day][5:7]\ncheck(f\"[B-oneday] the first autumn day followed by frost is {day_label}, \"\n      f\"with lowest temperature {x_day:.1f} deg C\", y_day > 0)\ntau_day = np.linspace(x_day - 3.0, x_day + 7.0, 1001)\nmg_day = y_day * (tau_day - x_day)\nzo_day, lg_day, hg_day = zo_loss(mg_day), log2_loss(mg_day), hinge_loss(mg_day)\ncheck(\"[B-oneday] its 0/1 loss is 1 for every threshold at or below the \"\n      \"day's temperature and 0 above it\",\n      np.all(zo_day[tau_day <= x_day] == 1.0)\n      and np.all(zo_day[tau_day > x_day] == 0.0))\ncheck(\"[B-oneday] the logistic loss / ln 2 and the hinge loss both equal 1 \"\n      \"at the jump (margin 0)\",\n      abs(log2_loss(0.0) - 1.0) < 1e-12 and hinge_loss(0.0) == 1.0)\ncheck(\"[B-oneday] both surrogates lie on or above the 0/1 loss at every \"\n      \"threshold and decrease with it\",\n      np.all(lg_day >= zo_day - 1e-12) and np.all(hg_day >= zo_day - 1e-12)\n      and np.all(np.diff(lg_day) < 0) and np.all(np.diff(hg_day) <= 0))\ncheck(f\"[B-oneday] one degree above the jump the 0/1 loss is 0 while the \"\n      f\"logistic loss / ln 2 is {log2_loss(1.0):.3f} and the hinge loss 0\",\n      zo_loss(1.0) == 0.0 and 0.4 < log2_loss(1.0) < 0.5 and hinge_loss(1.0) == 0)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-stair]** The average 0/1 loss over the 365 days is a staircase in tau: constant between consecutive distinct temperatures, one minus the accuracy, with the never-frost and always-frost baselines at its two ends, a flat interval of minimizers, and a difference quotient of exactly zero away from the jumps."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"[B-stair] the average 0/1 loss over the year is a staircase\")\nxs = np.unique(x)                               # the jump positions\nn_steps = len(xs)\n# value of the staircase on the open interval (xs[k], xs[k+1]); the last\n# entry is the value above the largest temperature\nmids = np.concatenate([(xs[:-1] + xs[1:]) / 2.0, [xs[-1] + 1.0]])\nstair = np.array([avg_zo(t) for t in mids])\nbelow = avg_zo(xs[0] - 1.0)                     # never predict frost\ncheck(f\"[B-stair] the staircase has {n_steps} jumps, one per distinct \"\n      f\"temperature among the {m} days\", n_steps < m)\nprobe = np.concatenate([(xs[:-1] + 0.25 * (xs[1:] - xs[:-1])),\n                        (xs[:-1] + 0.75 * (xs[1:] - xs[:-1]))])\ncheck(\"[B-stair] between two consecutive temperatures the average 0/1 loss \"\n      \"is constant\",\n      all(avg_zo(t) == stair[k] for k, t in enumerate(probe[: n_steps - 1]))\n      and all(avg_zo(t) == stair[k]\n              for k, t in enumerate(probe[n_steps - 1:])))\nacc_at = [np.mean(np.where(tau - x > 0, 1.0, -1.0) == y) for tau in mids]\ncheck(\"[B-stair] it equals one minus the accuracy at every threshold\",\n      np.allclose(stair, 1.0 - np.array(acc_at)))\ncheck(f\"[B-stair] below every temperature the warning never predicts frost \"\n      f\"and the average 0/1 loss is the frost fraction {below:.3f}; above \"\n      f\"every temperature it always predicts frost and the loss is \"\n      f\"{stair[-1]:.3f}\",\n      abs(below - n_frost / m) < 1e-12 and abs(stair[-1] - (1 - n_frost / m)) < 1e-12)\nzo_min = float(stair.min())\nflat = np.flatnonzero(stair == zo_min)\n# the minimizers form open intervals between consecutive temperatures;\n# adjacent steps are collected into runs, each named by its two ends\nruns, start = [], flat[0]\nfor a, b in zip(flat, flat[1:]):\n    if b != a + 1:\n        runs.append((float(xs[start]), float(xs[a + 1])))\n        start = b\nruns.append((float(xs[start]), float(xs[flat[-1] + 1])))\nlo_tau, hi_tau = runs[0][0], runs[-1][1]\nspans = \" and \".join(f\"({a:.1f}, {b:.1f})\" for a, b in runs)\ncheck(f\"[B-stair] the smallest average 0/1 loss is {zo_min:.3f} \"\n      f\"({int(round(zo_min * m))} of {m} days wrong, accuracy \"\n      f\"{1 - zo_min:.3f}), reached by every threshold in the open \"\n      f\"interval(s) {spans} deg C: {len(runs)} interval(s)\",\n      zo_min < below and all(b > a for a, b in runs))\ncheck(f\"[B-stair] a threshold of exactly {lo_tau:.1f} deg C does worse \"\n      f\"({avg_zo(lo_tau):.3f}): a day at margin 0 counts as misclassified\",\n      avg_zo(lo_tau) > zo_min)\ngrid = np.linspace(-12.0, 20.0, 641)\naway = grid[np.min(np.abs(grid[:, None] - xs[None, :]), axis=1) > 1e-3]\ndelta = 5e-4\ncheck(\"[B-stair] the difference quotient of the average 0/1 loss is exactly \"\n      \"zero at every grid point away from the jumps\",\n      all(avg_zo(t + delta) - avg_zo(t) == 0.0 for t in away))\ncheck(f\"[B-stair] the staircase takes {len(np.unique(stair))} distinct \"\n      f\"values, each a multiple of 1/{m}\",\n      np.allclose(stair * m, np.round(stair * m)))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-smooth]** The average logistic loss / ln 2 and the average hinge loss are convex in tau and lie above the staircase; each has one minimizer, and the average 0/1 loss there is compared with the smallest average 0/1 loss."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "def argmin_convex(f, lo, hi, iters=200):\n    \"\"\"Ternary search for the minimizer of a convex function on [lo, hi].\"\"\"\n    for _ in range(iters):\n        a, b = lo + (hi - lo) / 3.0, hi - (hi - lo) / 3.0\n        if f(a) < f(b):\n            hi = b\n        else:\n            lo = a\n    return (lo + hi) / 2.0\n\n\nprint(\"[B-smooth] the average logistic and hinge losses are convex curves\")\nlg_grid = np.array([avg_log2(t) for t in grid])\nhg_grid = np.array([avg_hinge(t) for t in grid])\nzo_grid = np.array([avg_zo(t) for t in grid])\ncheck(\"[B-smooth] both averages are convex in the threshold (second \"\n      \"differences nonnegative on the grid)\",\n      np.all(np.diff(lg_grid, 2) >= -1e-10)\n      and np.all(np.diff(hg_grid, 2) >= -1e-10))\ncheck(\"[B-smooth] both averages lie on or above the average 0/1 loss at \"\n      \"every grid point\",\n      np.all(lg_grid >= zo_grid - 1e-12) and np.all(hg_grid >= zo_grid - 1e-12))\ntau_log = argmin_convex(avg_log2, -12.0, 20.0)\ntau_hinge = argmin_convex(avg_hinge, -12.0, 20.0)\ncheck(f\"[B-smooth] the average logistic loss / ln 2 has its minimum \"\n      f\"{avg_log2(tau_log):.3f} at tau = {tau_log:.2f} deg C, where the \"\n      f\"average 0/1 loss is {avg_zo(tau_log):.3f}\",\n      lo_tau - 3.0 < tau_log < hi_tau + 3.0)\ncheck(f\"[B-smooth] the average hinge loss has its minimum \"\n      f\"{avg_hinge(tau_hinge):.3f} at tau = {tau_hinge:.2f} deg C, where \"\n      f\"the average 0/1 loss is {avg_zo(tau_hinge):.3f}\",\n      lo_tau - 3.0 < tau_hinge < hi_tau + 3.0)\ncheck(f\"[B-smooth] the staircase at the two surrogate minimizers \"\n      f\"({avg_zo(tau_log):.3f}, {avg_zo(tau_hinge):.3f}) is within \"\n      f\"{3 / m:.3f} (three days) of its minimum {zo_min:.3f}\",\n      avg_zo(tau_log) <= zo_min + 3 / m + 1e-12\n      and avg_zo(tau_hinge) <= zo_min + 3 / m + 1e-12)\ncheck(\"[B-smooth] the slope of the average logistic loss is negative one \"\n      \"degree below its minimizer and positive one degree above it\",\n      avg_log2(tau_log - 1.0 + 1e-3) < avg_log2(tau_log - 1.0)\n      and avg_log2(tau_log + 1.0 + 1e-3) > avg_log2(tau_log + 1.0))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-scale]** Multiplying the hypothesis by a positive factor leaves the average 0/1 loss unchanged and changes the average logistic loss."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"[B-scale] the hypothesis multiplied by a positive factor\")\ncheck(\"[B-scale] the average 0/1 loss is the same for h, h/2 and 2h at \"\n      \"every grid point\",\n      all(avg_zo(t, 0.5) == avg_zo(t) == avg_zo(t, 2.0) for t in grid))\ncheck(f\"[B-scale] the average logistic loss / ln 2 at tau = {tau_log:.2f} \"\n      f\"changes: {avg_log2(tau_log, 0.5):.3f} for h/2, \"\n      f\"{avg_log2(tau_log):.3f} for h, {avg_log2(tau_log, 2.0):.3f} for 2h\",\n      avg_log2(tau_log, 0.5) != avg_log2(tau_log)\n      != avg_log2(tau_log, 2.0))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-plot]** CSVs for the figure and the matplotlib preview."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"[B-plot] CSVs and preview\")\nwith open(OUT_DIR / \"zerooneloss_oneday.csv\", \"w\") as fh:\n    fh.write(\"tau,zo,log2,hinge\\n\")\n    for t, a, b, c in zip(tau_day, zo_day, lg_day, hg_day):\n        fh.write(f\"{t:.4f},{a:.0f},{b:.6f},{c:.6f}\\n\")\nwith open(OUT_DIR / \"zerooneloss_stair.csv\", \"w\") as fh:\n    fh.write(\"tau,zo\\n\")\n    fh.write(f\"{grid[0]:.4f},{below:.6f}\\n\")\n    for t, v in zip(xs, stair):\n        fh.write(f\"{t:.4f},{v:.6f}\\n\")\n    fh.write(f\"{grid[-1]:.4f},{stair[-1]:.6f}\\n\")\nwith open(OUT_DIR / \"zerooneloss_curves.csv\", \"w\") as fh:\n    fh.write(\"tau,log2,hinge\\n\")\n    for t, a, b in zip(grid, lg_grid, hg_grid):\n        fh.write(f\"{t:.4f},{a:.6f},{b:.6f}\\n\")\ncheck(f\"[B-plot] the staircase CSV has {n_steps + 2} rows, the curves CSV \"\n      f\"{len(grid)}\", True)\n\nfig, (ax1, ax2) = plt.subplots(1, 2, figsize=(9.6, 3.9))\nax1.step(tau_day, zo_day, where=\"post\", color=\"black\", lw=1.6,\n         label=\"0/1 loss\")\nax1.plot(tau_day, lg_day, \"k--\", lw=1.3, label=\"logistic loss / ln 2\")\nax1.plot(tau_day, hg_day, \"k:\", lw=1.6, label=\"hinge loss\")\nax1.set_xlabel(\"threshold tau (deg C)\")\nax1.set_ylabel(f\"loss of the day {day_label}\")\nax1.set_title(f\"one day: tmin = {x_day:.1f} deg C, frost on the next day\")\nax1.set_ylim(-0.05, 3.2)\nax1.legend(frameon=False, fontsize=8)\nstair_x = np.concatenate([[grid[0]], xs, [grid[-1]]])\nstair_y = np.concatenate([[below], stair, [stair[-1]]])\nax2.step(stair_x, stair_y, where=\"post\", color=\"black\", lw=1.6,\n         label=\"0/1 loss\")\nax2.plot(grid, lg_grid, \"k--\", lw=1.3, label=\"logistic loss / ln 2\")\nax2.plot(grid, hg_grid, \"k:\", lw=1.6, label=\"hinge loss\")\nax2.set_xlabel(\"threshold tau (deg C)\")\nax2.set_ylabel(f\"average loss over the {m} days\")\nax2.set_title(\"all days: a staircase against two convex curves\")\nax2.set_xlim(grid[0], grid[-1])\nax2.set_ylim(0.0, 1.6)\nax2.legend(frameon=False, fontsize=8, loc=\"upper right\",\n           bbox_to_anchor=(0.98, 0.92))\nfig.tight_layout()\nfig.savefig(OUT_DIR / \"zerooneloss.png\", dpi=110)\n\nn_ok = sum(ok for _, ok in report)\nprint(f\"\\n{n_ok}/{len(report)} checks pass\")\nif n_ok != len(report):\n    raise SystemExit(1)"
  }
 ]
}