{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3",
   "language": "python"
  },
  "language_info": {
   "name": "python"
  },
  "colab": {
   "name": "hypothesis.ipynb"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# hypothesis \u2014 Python demo\n\nNumerical companion to the entry [hypothesis](https://dictionaryofml.org/terms/hypothesis.html) of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org/): it recomputes what the entry states and prints one line per check.\n\nOne block per paragraph of the entry (marked [P...]): each block verifies numerically what the corresponding statement asserts. Self-contained (numpy/matplotlib only), fixed seed.\n\nRequires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Generated from [`pythondemos/hypothesis.py`](https://dictionaryofml.org/terms/hypothesis.py); CC BY 4.0."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Notebook shim: the script resolves output paths relative to __file__,\n# which a notebook kernel does not define; everything lands in the\n# working directory instead.\nimport os\n__file__ = os.path.join(os.getcwd(), \"hypothesis.py\")\nos.makedirs(\"pythondemos\", exist_ok=True)"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "\"\"\"\nhypothesis.py \u2014 numerical companion to the glossary entry 'hypothesis'.\n\nOne block per paragraph of the entry (marked [P...]): each block verifies\nnumerically what the corresponding statement asserts. Self-contained\n(numpy/matplotlib only), fixed seed.\n\nBlocks\n------\n[P-def]   A hypothesis is a map h: X -> Y: a hypothesis for temperature\n          prediction returns a prediction y-hat = h(x) for every feature\n          value, and the same features always yield the same prediction.\n[P-learn] ML searches a SUBSET of Y^X: for finite spaces |X| = 4,\n          |Y| = 2 the set of all maps has |Y|^|X| = 16 elements, while\n          the subset of threshold maps has only 5 \u2014 restricting the\n          search is what makes learning with finite resources possible.\n          ERM over the trainset returns the element of the subset with\n          the smallest average loss on the trainset, achieving y ~ h(x).\n[P-repr]  Different hypothesis spaces represent their maps differently:\n          the same underlying map is represented as a polynomial\n          coefficient vector, as an executable Python function, and as\n          a decision-tree flow chart \u2014 all three agree on every input.\n\nOutputs\n-------\nhypothesis.png : preview figure (checking only).\n\nData generated by pythondemos/hypothesis.py.\n\"\"\"\n\nimport numpy as np\nimport matplotlib\n\nmatplotlib.use(\"Agg\")\nimport matplotlib.pyplot as plt\nfrom pathlib import Path\n\nOUT_DIR = Path(__file__).parent\n\nrng = np.random.default_rng(42)\nreport = []\n\n\ndef check(name, ok):\n    report.append((name, bool(ok)))\n    print(f\"  [{'ok' if ok else 'FAIL'}] {name}\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[P-def]** A hypothesis is a map h: X -> Y: a hypothesis for temperature prediction returns a prediction y-hat = h(x) for every feature value, and the same features always yield the same prediction."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"[P-def] a hypothesis maps features to predictions\")\nh = lambda x: 0.9 * x + 4.0                      # tomorrow ~ h(morning temp)\nx_morning = np.array([8.0, 12.0, 15.0])\ny_hat = h(x_morning)\ncheck(\"h returns a prediction for every feature value\",\n      y_hat.shape == x_morning.shape)\ncheck(\"h is a map: same features, same prediction\",\n      np.array_equal(h(x_morning), y_hat))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[P-learn]** ML searches a SUBSET of Y^X: for finite spaces |X| = 4, |Y| = 2 the set of all maps has |Y|^|X| = 16 elements, while the subset of threshold maps has only 5 \u2014 restricting the search is what makes learning with finite resources possible. ERM over the trainset returns the element of the subset with the smallest average loss on the trainset, achieving y ~ h(x)."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"[P-learn] the search is restricted to a subset of Y^X\")\nX_space = [0, 1, 2, 3]                            # |X| = 4\nY_space = [0, 1]                                  # |Y| = 2\nfrom itertools import product\nall_maps = list(product(Y_space, repeat=len(X_space)))\nthresh_maps = [tuple(int(x >= t) for x in X_space) for t in range(5)]\ncheck(\"|Y^X| = |Y|^|X| = 16 maps\", len(all_maps) == 2 ** 4 == 16)\ncheck(\"the threshold hypothesis space is a strict subset (5 of 16)\",\n      len(set(thresh_maps)) == 5\n      and set(thresh_maps) <= set(all_maps))\n# ERM over the subset achieves y ~ h(x)\nxs = rng.choice(X_space, 40)\nys = (xs >= 2).astype(int)                        # true threshold t = 2\nemp = [np.mean([m[x] != y for x, y in zip(xs, ys)]) for m in thresh_maps]\nh_hat = thresh_maps[int(np.argmin(emp))]\ncheck(\"ERM over the subset finds the true threshold map\",\n      h_hat == tuple(int(x >= 2) for x in X_space))\ncheck(\"the learned hypothesis achieves y = h(x) on the trainset\",\n      min(emp) == 0)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[P-repr]** Different hypothesis spaces represent their maps differently: the same underlying map is represented as a polynomial coefficient vector, as an executable Python function, and as a decision-tree flow chart \u2014 all three agree on every input."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"[P-repr] one map, three representations\")\nw_poly = np.array([1.0, -2.0, 0.5])               # h(x) = 1 - 2x + 0.5 x^2\nh_poly = lambda x: sum(w * x**j for j, w in enumerate(w_poly))\ndef h_code(x):                                    # executable source code\n    return 1.0 - 2.0 * x + 0.5 * x * x\ndef h_tree(x):                                    # flow chart of comparisons\n    # piecewise-constant approximation on a grid: a depth-3 tree\n    return h_poly(np.round(x * 8) / 8)\ngrid = np.linspace(-2, 2, 33)                     # tree grid points\ncheck(\"polynomial and source-code representations agree everywhere\",\n      np.allclose(h_poly(grid), h_code(grid)))\ncheck(\"the flow-chart (tree) representation agrees on its grid\",\n      np.allclose(h_tree(grid), h_poly(grid)))\n\n# ------------------------------------------------------------ preview\nfig, ax = plt.subplots(figsize=(4.8, 3.2))\nxx = np.linspace(-2, 2, 200)\nax.plot(xx, h_poly(xx), label=\"polynomial / code\")\nax.step(xx, h_tree(xx), where=\"mid\", ls=\"--\", label=\"tree (piecewise)\")\nax.legend(frameon=False)\nax.set_xlabel(\"feature $x$\"); ax.set_ylabel(\"prediction $h(x)$\")\nax.set_title(\"[P-repr] one hypothesis, several representations\")\nfig.tight_layout()\nfig.savefig(OUT_DIR / \"hypothesis.png\", dpi=110)\nprint(f\"\\n{sum(ok for _, ok in report)}/{len(report)} checks passed\")\nassert all(ok for _, ok in report)"
  }
 ]
}