{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3",
   "language": "python"
  },
  "language_info": {
   "name": "python"
  },
  "colab": {
   "name": "cm.ipynb"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# confusion matrix \u2014 Python demo\n\nNumerical companion to the entry [confusion matrix](https://dictionaryofml.org/terms/cm.html) of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org/): it recomputes what the entry states and prints one line per check.\n\nNumerical companion to the glossary entry 'cm' (confusion matrix). The GeoSphere Austria weather station Krems (station id 3805, 48.42 N, 15.62 E) records the minimum and maximum air temperature of each day; this script downloads the records for February and April 2024 from the GeoSphere data hub (dataset klima-v2-1d) and writes them to cm_weather.csv, so the exact numbers behind every figure stay on record. Checks pin the downloaded values to the 2024 archive, so a change on the server side is caught rather than silently absorbed.\n\nRequires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Generated from [`pythondemos/cm.py`](https://dictionaryofml.org/terms/cm.py); CC BY 4.0."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Notebook shim: the script resolves output paths relative to __file__,\n# which a notebook kernel does not define; everything lands in the\n# working directory instead.\nimport os\n__file__ = os.path.join(os.getcwd(), \"cm.py\")\nos.makedirs(\"pythondemos\", exist_ok=True)"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "\"\"\"Frost warning at Krems an der Donau: two classifiers with the same\naccuracy, told apart only by their confusion matrices.\n\nPurpose\n-------\nNumerical companion to the glossary entry 'cm' (confusion matrix).  The\nGeoSphere Austria weather station Krems (station id 3805, 48.42 N,\n15.62 E) records the minimum and maximum air temperature of each day;\nthis script downloads the records for February and April 2024 from the\nGeoSphere data hub (dataset klima-v2-1d) and writes them to\ncm_weather.csv, so the exact numbers behind every figure stay on\nrecord.  Checks pin the downloaded values to the 2024 archive, so a\nchange on the server side is caught rather than silently absorbed.\n\nEach of the 40 data points is one day: its features are the two\ntemperatures of that day, its label is whether the minimum temperature\nof the FOLLOWING day stays above 0 degrees C.  34 of the 40 following\ndays stay above 0 degrees C and only 6 bring frost, so the constant\nprediction \"above 0\" -- a baseline that ignores the features --\nis correct on 34 of the 40 days: accuracy 0.85.  A linear classifier\nlearned by logistic regression from the 40 data points reaches the same\naccuracy 0.85.  The two confusion matrices nevertheless differ: the\nlearned classifier detects one of the six frost days at the cost of one\nfalse alarm, while the baseline detects none.\n\nThe two 20-day windows (February 9-28 and April 10-29, each day paired\nwith the following day) are chosen so that each month contributes three\nfrost days: February 2024 was exceptionally warm, and the April frosts\nfall in the cold snap of April 19-26.\n\nDeterministic: no randomness (the minimization starts from the zero\nvector).  Self-contained: numpy + matplotlib only (stdlib urllib for\nthe download).\n\nBlocks\n------\n[B-fetch] Download the daily minimum and maximum temperature at Krems\n          for February and April 2024 from the GeoSphere data hub;\n          write the 59 records to cm_weather.csv and check them\n          against the archive.\n[B-data]  Build the 40 data points from the downloaded temperatures;\n          check the label counts: 34 next days above 0 degrees C,\n          6 with frost.\n[B-learn] Learn a linear classifier from the 40 data points by logistic\n          regression; check that the minimization has converged.\n[B-cm]    Confusion matrices of the learned classifier and of the\n          always-above-0 baseline; check that both reach accuracy 0.85\n          and that only the confusion matrices distinguish them.\n\nOutputs\n-------\ncm_weather.csv      : date, tmin, tmax of the 59 downloaded days\ncm_points_above.csv : tmin, tmax of the 34 days followed by a day above 0\ncm_points_frost.csv : tmin, tmax of the 6 days followed by a frost day\ncm_boundary.csv     : tmin, tmax along the learned decision boundary\ncm_counts.csv       : the four entries of both confusion matrices\ncm.png              : preview (checking only) -- the 40 data points with\n                      the decision boundary, and the two confusion\n                      matrices\n\"\"\"\n\nimport json\nimport urllib.request\nfrom pathlib import Path\n\nimport numpy as np\nimport matplotlib\n\nmatplotlib.use(\"Agg\")\nimport matplotlib.pyplot as plt\n\nOUT_DIR = Path(__file__).parent\n\nreport = []                         # collects (check name, pass/fail) pairs\n\n\ndef check(name, ok):                # records and prints one verification\n    report.append((name, bool(ok)))\n    print(f\"  [{'ok' if ok else 'FAIL'}] {name}\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-fetch]** Download the daily minimum and maximum temperature at Krems for February and April 2024 from the GeoSphere data hub; write the 59 records to cm_weather.csv and check them against the archive."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "URL = (\"https://dataset.api.hub.geosphere.at/v1/station/historical/\"\n       \"klima-v2-1d?parameters=tlmin,tlmax&station_ids=3805\"\n       \"&start=2024-02-01&end=2024-04-30\")\nwith urllib.request.urlopen(URL, timeout=120) as resp:\n    payload = json.load(resp)\nparams = payload[\"features\"][0][\"properties\"][\"parameters\"]\nrecords = [(stamp[:10], lo, hi)\n           for stamp, lo, hi in zip(payload[\"timestamps\"],\n                                    params[\"tlmin\"][\"data\"],\n                                    params[\"tlmax\"][\"data\"])\n           if stamp[5:7] in (\"02\", \"04\")]        # February and April only\nwith open(OUT_DIR / \"cm_weather.csv\", \"w\") as f:\n    f.write(\"date,tmin,tmax\\n\")\n    for day, lo, hi in records:\n        f.write(f\"{day},{lo},{hi}\\n\")\ncheck(\"[B-fetch] 59 days downloaded for February and April 2024\",\n      len(records) == 59)\ncheck(\"[B-fetch] the record matches the archive (Feb 1: -3.8 to 10.4)\",\n      records[0] == (\"2024-02-01\", -3.8, 10.4))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-data]** Build the 40 data points from the downloaded temperatures; check the label counts: 34 next days above 0 degrees C, 6 with frost."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "dates = [day for day, _, _ in records]\ntemps = np.array([[lo, hi] for _, lo, hi in records])\ndate_index = {d: i for i, d in enumerate(dates)}\n\npredictor_days = ([date_index[\"2024-02-09\"] + k for k in range(20)] +\n                  [date_index[\"2024-04-10\"] + k for k in range(20)])\nidx = np.array(predictor_days)\n\nX = temps[idx]                            # features: (tmin, tmax) of day t\ny = np.where(temps[idx + 1, 0] > 0.0, 1.0, -1.0)   # +1: next day above 0\nabove, frost = y > 0, y < 0\n\ncheck(\"[B-data] 40 data points, 20 per month\", len(y) == 40)\ncheck(\"[B-data] 34 next days stay above 0 degrees C\", int(above.sum()) == 34)\ncheck(\"[B-data] 6 next days bring frost, 3 per month\",\n      int(frost.sum()) == 6 and int(frost[:20].sum()) == 3)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-learn]** Learn a linear classifier from the 40 data points by logistic regression; check that the minimization has converged."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "Xb = np.hstack([X, np.ones((len(y), 1))])          # append constant feature\nw = np.zeros(3)\nSTEP, ITERS = 0.01, 200_000\nfor _ in range(ITERS):\n    grad = -(y[:, None] * Xb / (1 + np.exp(y * (Xb @ w)))[:, None]).mean(0)\n    w -= STEP * grad\ncheck(\"[B-learn] the minimization has converged (tiny final update)\",\n      float(np.linalg.norm(grad)) < 1e-4)\nprint(f\"  learned weights: {w[0]:+.3f} * tmin {w[1]:+.3f} * tmax {w[2]:+.3f}\")\n\npred = np.where(Xb @ w > 0, 1.0, -1.0)\npred_baseline = np.ones(len(y))                    # always \"above 0\"\n\n\ndef confusion(y_true, y_pred):\n    \"\"\"2x2 counts; rows: true above 0 / frost, columns: predicted.\"\"\"\n    return np.array([[int(np.sum((y_true == a) & (y_pred == p)))\n                      for p in (1.0, -1.0)] for a in (1.0, -1.0)])\n\n\ncm_clf = confusion(y, pred)\ncm_base = confusion(y, pred_baseline)\nacc_clf = float(np.trace(cm_clf)) / len(y)\nacc_base = float(np.trace(cm_base)) / len(y)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-cm]** Confusion matrices of the learned classifier and of the always-above-0 baseline; check that both reach accuracy 0.85 and that only the confusion matrices distinguish them."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(f\"  learned classifier: {cm_clf.tolist()}  accuracy {acc_clf:.2f}\")\nprint(f\"  baseline:           {cm_base.tolist()}  accuracy {acc_base:.2f}\")\ncheck(\"[B-cm] both reach accuracy 0.85\",\n      abs(acc_clf - 0.85) < 1e-9 and abs(acc_base - 0.85) < 1e-9)\ncheck(\"[B-cm] the learned classifier detects a frost day, one false alarm\",\n      cm_clf[1, 1] == 1 and cm_clf[0, 1] == 1)\ncheck(\"[B-cm] the baseline detects no frost day at all\", cm_base[1, 1] == 0)\n\n# ---- outputs: CSVs for the entry's pgfplots figure, preview PNG\nheader = \"tmin,tmax\"\nnp.savetxt(OUT_DIR / \"cm_points_above.csv\", X[above], delimiter=\",\",\n           header=header, comments=\"\", fmt=\"%.1f\")\nnp.savetxt(OUT_DIR / \"cm_points_frost.csv\", X[frost], delimiter=\",\",\n           header=header, comments=\"\", fmt=\"%.1f\")\nt_line = np.linspace(-5.0, 14.0, 2)\nboundary = np.stack([t_line, -(w[0] * t_line + w[2]) / w[1]], 1)\nnp.savetxt(OUT_DIR / \"cm_boundary.csv\", boundary, delimiter=\",\",\n           header=header, comments=\"\", fmt=\"%.3f\")\nwith open(OUT_DIR / \"cm_counts.csv\", \"w\") as f:\n    f.write(\"classifier,true,pred_above,pred_frost\\n\")\n    for name, cm in ((\"learned\", cm_clf), (\"baseline\", cm_base)):\n        for row, true in zip(cm, (\"above\", \"frost\")):\n            f.write(f\"{name},{true},{row[0]},{row[1]}\\n\")\n\nfig, axes = plt.subplots(1, 3, figsize=(11, 3.4),\n                         gridspec_kw={\"width_ratios\": [1.6, 1, 1]})\nax = axes[0]\nax.plot(X[above, 0], X[above, 1], \"o\", color=\"tab:blue\", ms=5,\n        label=\"next day above 0 \u00b0C\")\nax.plot(X[frost, 0], X[frost, 1], \"^\", mfc=\"none\", mec=\"tab:red\", ms=8,\n        mew=1.5, label=\"next day frost\")\nax.plot(boundary[:, 0], boundary[:, 1], \"k--\", lw=1.2,\n        label=\"decision boundary\")\nax.set_xlabel(\"minimum temperature of the day (\u00b0C)\")\nax.set_ylabel(\"maximum temperature of the day (\u00b0C)\")\nax.set_ylim(4, 31)\nax.set_title(\"40 days at Krems, Feb/Apr 2024\")\nax.legend(frameon=False, fontsize=8)\n\nfor ax, name, cm, acc in ((axes[1], \"learned classifier\", cm_clf, acc_clf),\n                          (axes[2], \"always-above-0 baseline\", cm_base,\n                           acc_base)):\n    ax.imshow(cm, cmap=\"Greys\", vmin=0, vmax=45)\n    for i in range(2):\n        for j in range(2):\n            ax.text(j, i, str(cm[i, j]), ha=\"center\", va=\"center\",\n                    color=\"black\" if cm[i, j] < 25 else \"white\")\n    ax.set_xticks([0, 1], [\"above 0\", \"frost\"])\n    ax.set_yticks([0, 1], [\"above 0\", \"frost\"])\n    ax.set_xlabel(\"predicted\")\n    ax.set_ylabel(\"true\")\n    ax.set_title(f\"{name}\\naccuracy {acc:.2f}\")\n\nfig.tight_layout()\nfig.savefig(OUT_DIR / \"cm.png\", dpi=150)\n\nfailed = [name for name, ok in report if not ok]\nprint(f\"{len(report) - len(failed)}/{len(report)} checks passed\"\n      + (f\", FAILED: {failed}\" if failed else \"\"))"
  }
 ]
}