{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3",
   "language": "python"
  },
  "language_info": {
   "name": "python"
  },
  "colab": {
   "name": "trainset.ipynb"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# training set \u2014 Python demo\n\nNumerical companion to the entry [training set](https://dictionaryofml.org/terms/trainset.html) of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org/): it recomputes what the entry states and prints one line per check.\n\nThe entry's weather narrative, carried out: three days of synthetic weather recordings (morning minimum temperature as feature, maximum daytime temperature as label) form the training set, and two hypotheses are learned from it -- a straight line and a degree-two polynomial. Every claim of the entry is measured on the result. Self-contained (numpy/matplotlib only), deterministic.\n\nRequires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Generated from [`pythondemos/trainset.py`](https://dictionaryofml.org/terms/trainset.py); CC BY 4.0."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Notebook shim: the script resolves output paths relative to __file__,\n# which a notebook kernel does not define; everything lands in the\n# working directory instead.\nimport os\n__file__ = os.path.join(os.getcwd(), \"trainset.py\")\nos.makedirs(\"pythondemos\", exist_ok=True)"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "\"\"\"\ntrainset.py -- numerical companion to the entry 'training set'.\n\nThe entry's weather narrative, carried out: three days of synthetic weather\nrecordings (morning minimum temperature as feature, maximum daytime\ntemperature as label) form the training set, and two hypotheses are learned\nfrom it -- a straight line and a degree-two polynomial. Every claim of the\nentry is measured on the result. Self-contained (numpy/matplotlib only),\ndeterministic.\n\nBlocks\n------\n[B-trainset]   Three days of recordings form the training set: for each day\n               the morning minimum temperature (feature) and the maximum\n               daytime temperature (label) are known, so the loss of any\n               candidate hypothesis can be evaluated on them.\n[B-erm]        Fit a straight line by minimizing the average squared error\n               over the training set. The value of that minimum is the\n               training error, and any other line incurs a larger average.\n[B-poly]       Fit a degree-two polynomial to the same training set: it\n               passes through all three data points, so its training error\n               is zero -- smaller than the training error of the line.\n[B-misleading] A small training error can be misleading: on twenty held-back\n               validation days, the polynomial's average loss (its validation\n               error) far exceeds the line's, reversing the training-set\n               ranking (overfitting).\n[B-picture]    The training set and both hypotheses evaluated on a grid,\n               written to CSV for the entry's figure.\n\nOutputs\n-------\npythondemos/trainset.png        : preview figure (checking only).\npythondemos/trainset_train.csv  : the three training days (x = morning\n                                  minimum temperature, y = maximum daytime\n                                  temperature)\npythondemos/trainset_line.csv   : the fitted line on a two-point grid\npythondemos/trainset_poly.csv   : the fitted polynomial on a dense grid\n\"\"\"\n\nimport numpy as np\nimport matplotlib\n\nmatplotlib.use(\"Agg\")\nimport matplotlib.pyplot as plt\n\nfrom pathlib import Path\n\nOUT_DIR = Path(__file__).parent\n\nreport = []\n\n\ndef check(name, ok):\n    report.append((name, bool(ok)))\n    print(f\"  [{'ok' if ok else 'FAIL'}] {name}\")\n\n\ndef days(n, gen, lo=-15.0, hi=5.0, w1=0.8, w0=4.0, noise=2.0):\n    \"\"\"n days: morning minimum x and maximum daytime temperature y.\"\"\"\n    x = np.sort(gen.uniform(lo, hi, n))\n    y = w0 + w1 * x + gen.normal(0.0, noise, n)\n    return x, y\n\n\ndef avg_sqerr(x, y, coeffs):\n    return float(np.mean((y - np.polyval(coeffs, x)) ** 2))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-trainset]** Three days of recordings form the training set: for each day the morning minimum temperature (feature) and the maximum daytime temperature (label) are known, so the loss of any candidate hypothesis can be evaluated on them."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"[B-trainset] three days of recordings form the training set\")\n\nx, y = days(3, np.random.default_rng(100))\nfor xr, yr in zip(x, y):\n    print(f\"    morning minimum {xr:6.1f} C, maximum daytime {yr:6.1f} C\")\ncheck(\"both temperatures of every training day are known\",\n      np.all(np.isfinite(x)) and np.all(np.isfinite(y)))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-erm]** Fit a straight line by minimizing the average squared error over the training set. The value of that minimum is the training error, and any other line incurs a larger average."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"\\n[B-erm] the line minimizes the average squared error over the \"\n      \"training set\")\n\nc_line = np.polyfit(x, y, 1)\ntrainerr_line = avg_sqerr(x, y, c_line)\nprint(f\"    fitted line: max temperature = {c_line[1]:.2f} + {c_line[0]:.2f}\"\n      f\" * min temperature; training error {trainerr_line:.3f}\")\ngen = np.random.default_rng(7)\nothers = [avg_sqerr(x, y, c_line + d) for d in gen.normal(0.0, 0.3, (200, 2))]\ncheck(\"every perturbed line incurs a larger average on the training set\",\n      all(o >= trainerr_line for o in others))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-poly]** Fit a degree-two polynomial to the same training set: it passes through all three data points, so its training error is zero -- smaller than the training error of the line."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"\\n[B-poly] a degree-two polynomial passes through all three days\")\n\nc_poly = np.polyfit(x, y, 2)\ntrainerr_poly = avg_sqerr(x, y, c_poly)\nprint(f\"    polynomial training error {trainerr_poly:.2e} \"\n      f\"(line: {trainerr_line:.3f})\")\ncheck(\"the polynomial's training error is zero (up to round-off)\",\n      trainerr_poly < 1e-10)\ncheck(\"the polynomial's training error is smaller than the line's\",\n      trainerr_poly < trainerr_line)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-misleading]** A small training error can be misleading: on twenty held-back validation days, the polynomial's average loss (its validation error) far exceeds the line's, reversing the training-set ranking (overfitting)."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"\\n[B-misleading] the training-set ranking of the two hypotheses is \"\n      \"misleading\")\n\nxv, yv = days(20, np.random.default_rng(1100))\nvalerr_line = avg_sqerr(xv, yv, c_line)\nvalerr_poly = avg_sqerr(xv, yv, c_poly)\nprint(f\"    validation error on 20 held-back days: line {valerr_line:.3f}, \"\n      f\"polynomial {valerr_poly:.3f}\")\ncheck(\"the polynomial's validation error far exceeds the line's\",\n      valerr_poly > 3 * valerr_line)\ncheck(\"the validation days are disjoint from the training days\",\n      len(set(np.round(xv, 6)) & set(np.round(x, 6))) == 0)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-picture]** The training set and both hypotheses evaluated on a grid, written to CSV for the entry's figure."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"\\n[B-picture] training set and both hypotheses, written to CSV\")\n\nxs_line = np.array([-14.0, 5.0])\nxs_poly = np.linspace(-14.0, 5.0, 200)\n\n\ndef write_csv(name, cols, header):\n    np.savetxt(OUT_DIR / name, np.column_stack(cols), delimiter=\",\",\n               header=header, comments=\"\", fmt=\"%.4f\")\n\n\nwrite_csv(\"trainset_train.csv\", (x, y), \"x,y\")\nwrite_csv(\"trainset_line.csv\", (xs_line, np.polyval(c_line, xs_line)),\n          \"x,yhat\")\nwrite_csv(\"trainset_poly.csv\", (xs_poly, np.polyval(c_poly, xs_poly)),\n          \"x,yhat\")\nprint(f\"    wrote {len(x)} training days, the line and the polynomial to \"\n      f\"3 CSV files\")\ncheck(\"the two curves agree on the training days only (max gap elsewhere \"\n      \"is large)\",\n      np.max(np.abs(np.polyval(c_poly, xs_poly)\n                    - np.polyval(c_line, xs_poly))) > 3.0)\n\nfig, ax = plt.subplots(figsize=(5.6, 3.6))\nax.plot(xs_poly, np.polyval(c_poly, xs_poly), \"--\", color=\"black\", lw=1.2,\n        label=\"polynomial (training error zero)\")\nax.plot(xs_line, np.polyval(c_line, xs_line), \"-\", color=\"black\", lw=1.2,\n        label=\"straight line\")\nax.plot(x, y, \"o\", ms=6, color=\"black\", label=\"training set\")\nax.set_xlabel(\"morning minimum temperature\")\nax.set_ylabel(\"maximum daytime temperature\")\nax.set_title(\"three training days, two hypotheses learned from them\",\n             fontsize=9)\nax.legend(frameon=False, fontsize=8)\n\nfig.tight_layout()\nfig.savefig(OUT_DIR / \"trainset.png\", dpi=110)\n\nn_ok = sum(ok for _, ok in report)\nprint(f\"\\n{n_ok}/{len(report)} checks pass\")\nprint(\"wrote trainset.png\")\nif n_ok != len(report):\n    raise SystemExit(1)"
  }
 ]
}