{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3",
   "language": "python"
  },
  "language_info": {
   "name": "python"
  },
  "colab": {
   "name": "finetuning.ipynb"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# fine-tuning \u2014 Python demo\n\nNumerical companion to the entry [fine-tuning](https://dictionaryofml.org/terms/finetuning.html) of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org/): it recomputes what the entry states and prints one line per check.\n\nThe entry's claims, measured: a linear hypothesis is pretrained on a large dataset from a related task, then adapted to a small task-specific training set by a few GD steps. Warm-starting at the pretrained model parameters keeps the iterates in a small neighborhood and reaches a small validation error; the same iteration from a fresh initialization overfits the small training set. Self-contained (numpy/matplotlib only), deterministic.\n\nRequires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Generated from [`pythondemos/finetuning.py`](https://dictionaryofml.org/terms/finetuning.py); CC BY 4.0."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Notebook shim: the script resolves output paths relative to __file__,\n# which a notebook kernel does not define; everything lands in the\n# working directory instead.\nimport os\n__file__ = os.path.join(os.getcwd(), \"finetuning.py\")\nos.makedirs(\"pythondemos\", exist_ok=True)"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "\"\"\"\nfinetuning.py -- numerical companion to the entry 'fine-tuning'.\n\nThe entry's claims, measured: a linear hypothesis is pretrained on a large\ndataset from a related task, then adapted to a small task-specific training\nset by a few GD steps. Warm-starting at the pretrained model\nparameters keeps the iterates in a small neighborhood and reaches a small\nvalidation error; the same iteration from a fresh initialization overfits\nthe small training set. Self-contained (numpy/matplotlib only),\ndeterministic.\n\nBlocks\n------\n[B-tasks]    Two related tasks with 30 model parameters: pretraining data\n             (2000 data points) from parameters w*, task data from\n             w* + delta with a small shift delta. Pretraining recovers\n             parameters close to w*.\n[B-warm]     Fine-tuning: GD on the task's small training\n             set (15 data points), warm-started at the pretrained\n             parameters, dips to a validation error below twice the noise\n             level within the first ~20 steps; running all 500 steps\n             interpolates the 15 points and drifts upward again -- the\n             measured reason fine-tuning stops early.\n[B-fresh]    The same iteration from a fresh (zero) initialization drives\n             the training error low but its validation error stays several\n             times larger: it overfits the 15 data points.\n[B-distance] The iterates stay near where they start: the distance moved\n             is bounded by the accumulated update lengths, and the warm start moves only about the length of the\n             task shift delta, far less than the fresh run.\n\nOutputs\n-------\npythondemos/finetuning.png : preview figure (checking only).\n\"\"\"\n\nimport numpy as np\nimport matplotlib\n\nmatplotlib.use(\"Agg\")\nimport matplotlib.pyplot as plt\n\nfrom pathlib import Path\n\nOUT_DIR = Path(__file__).parent\n\nreport = []\n\n\ndef check(name, ok):\n    report.append((name, bool(ok)))\n    print(f\"  [{'ok' if ok else 'FAIL'}] {name}\")\n\n\nD = 30\nNOISE = 0.5\ngen = np.random.default_rng(20260829)\n\nw_star = gen.normal(size=D)\nw_star *= 3.0 / np.linalg.norm(w_star)          # |w*| = 3\ndelta = gen.normal(size=D)\ndelta *= 0.5 / np.linalg.norm(delta)            # small task shift, |delta| = 0.5\nw_task = w_star + delta\n\n\ndef draw(n, w, seed):\n    g = np.random.default_rng(seed)\n    X = g.normal(size=(n, D))\n    y = X @ w + g.normal(0.0, NOISE, n)\n    return X, y\n\n\ndef avg_sqerr(X, y, w):\n    return float(np.mean((y - X @ w) ** 2))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-tasks]** Two related tasks with 30 model parameters: pretraining data (2000 data points) from parameters w*, task data from w* + delta with a small shift delta. Pretraining recovers parameters close to w*."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"[B-tasks] a large pretraining dataset and a small task dataset\")\n\nXp, yp = draw(2000, w_star, seed=1)\nw_pre = np.linalg.lstsq(Xp, yp, rcond=None)[0]\nXt, yt = draw(15, w_task, seed=2)\nXv, yv = draw(100_000, w_task, seed=3)\nprint(f\"    pretraining on {len(yp)} data points: |w_pre - w*| = \"\n      f\"{np.linalg.norm(w_pre - w_star):.3f} (|w*| = 3, task shift \"\n      f\"|delta| = 0.5)\")\nprint(f\"    task training set: {len(yt)} data points, {D} model parameters\")\ncheck(\"pretraining recovers parameters close to w*\",\n      np.linalg.norm(w_pre - w_star) < 0.15)\n\n\ndef run_gd(w0, steps=500, eta=0.02):\n    w = w0.copy()\n    dist, val, moved = [], [], 0.0\n    for t in range(steps):\n        g = -2.0 / len(yt) * Xt.T @ (yt - Xt @ w)\n        w = w - eta * g\n        moved += eta * float(np.linalg.norm(g))\n        dist.append(float(np.linalg.norm(w - w0)))\n        val.append(avg_sqerr(Xv, yv, w))\n    return w, dist, val, moved"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-warm]** Fine-tuning: GD on the task's small training set (15 data points), warm-started at the pretrained parameters, dips to a validation error below twice the noise level within the first ~20 steps; running all 500 steps interpolates the 15 points and drifts upward again -- the measured reason fine-tuning stops early."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"\\n[B-warm] warm-started GD on the 15 data points\")\n\nw_warm, dist_w, val_w, moved_w = run_gd(w_pre)\nbest = int(np.argmin(val_w))\nprint(f\"    validation error dips to {val_w[best]:.3f} at step {best + 1} \"\n      f\"(noise level {NOISE**2:.2f}), ends at {val_w[-1]:.3f} after \"\n      f\"interpolating the 15 points (training error \"\n      f\"{avg_sqerr(Xt, yt, w_warm):.3f})\")\ncheck(\"stopped at its best step, the warm start reaches a validation \"\n      \"error below twice the noise level\", val_w[best] < 2 * NOISE**2)\ncheck(\"running on drifts upward again -- the reason fine-tuning is \"\n      \"stopped after few steps\", val_w[-1] > 1.2 * val_w[best])"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-fresh]** The same iteration from a fresh (zero) initialization drives the training error low but its validation error stays several times larger: it overfits the 15 data points."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"\\n[B-fresh] the same iteration from a fresh initialization\")\n\nw_fresh, dist_f, val_f, moved_f = run_gd(np.zeros(D))\nprint(f\"    validation error {val_f[-1]:.3f}; training error \"\n      f\"{avg_sqerr(Xt, yt, w_fresh):.3f} (15 data points, 30 parameters)\")\ncheck(\"the fresh run drives the training error below the noise level\",\n      avg_sqerr(Xt, yt, w_fresh) < NOISE**2)\ncheck(\"yet its validation error is at least four times the warm start's\",\n      val_f[-1] > 4 * val_w[-1])"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-distance]** The iterates stay near where they start: the distance moved is bounded by the accumulated update lengths, and the warm start moves only about the length of the task shift delta, far less than the fresh run."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "print(\"\\n[B-distance] the iterates stay near where they start\")\n\nprint(f\"    distance moved: warm {dist_w[-1]:.2f} (bound \"\n      f\"eta*sum|grad| = {moved_w:.2f}), fresh {dist_f[-1]:.2f}; \"\n      f\"task shift |delta| = 0.5, |w*| = 3\")\ncheck(\"the distance moved is bounded by the accumulated step lengths\",\n      dist_w[-1] <= moved_w + 1e-9 and dist_f[-1] <= moved_f + 1e-9)\ncheck(\"the warm start moves on the scale of the task shift, far less \"\n      \"than the fresh run\", dist_w[-1] < 1.0 and dist_f[-1] > 2.0)\n\n\n# --------------------------------------------------------------- preview\nfig, ax = plt.subplots(1, 2, figsize=(9.2, 3.6))\nsteps = range(1, len(val_w) + 1)\nax[0].plot(steps, val_w, \"-\", color=\"black\", label=\"warm start (pretrained)\")\nax[0].plot(steps, val_f, \"--\", color=\"0.4\", label=\"fresh initialization\")\nax[0].axhline(NOISE**2, color=\"0.75\", lw=0.8)\nax[0].set_yscale(\"log\")\nax[0].set_xlabel(\"GD step\")\nax[0].set_ylabel(\"validation error\")\nax[0].set_title(\"[B-warm]/[B-fresh] 15 data points, 30 parameters\",\n                fontsize=9)\nax[0].legend(frameon=False, fontsize=8)\n\nax[1].plot(steps, dist_w, \"-\", color=\"black\", label=\"warm start\")\nax[1].plot(steps, dist_f, \"--\", color=\"0.4\", label=\"fresh initialization\")\nax[1].set_xlabel(\"GD step\")\nax[1].set_ylabel(\"distance from initialization\")\nax[1].set_title(\"[B-distance] how far the iterates move\", fontsize=9)\nax[1].legend(frameon=False, fontsize=8)\n\nfig.tight_layout()\nfig.savefig(OUT_DIR / \"finetuning.png\", dpi=110)\n\nn_ok = sum(ok for _, ok in report)\nprint(f\"\\n{n_ok}/{len(report)} checks pass\")\nprint(\"wrote finetuning.png\")\nif n_ok != len(report):\n    raise SystemExit(1)"
  }
 ]
}