{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3",
   "language": "python"
  },
  "language_info": {
   "name": "python"
  },
  "colab": {
   "name": "transferlearning.ipynb"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# transfer learning \u2014 Python demo\n\nNumerical companion to the entry [transfer learning](https://dictionaryofml.org/terms/transferlearning.html) of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org/): it recomputes what the entry states and prints one line per check.\n\nChecks the entry's claims on a small deep net with one-dimensional input: a hypothesis learned on a source task with many data points, whose early layer is kept and whose remaining model parameters are adapted by fine-tuning on a few data points of a target task, reaches a smaller risk on the target task than a hypothesis learned from those few data points alone; and when the features of the source task are not useful for the target task, the fine-tuned hypothesis has a larger risk than one learned from scratch (negative transfer). Self-contained (numpy and matplotlib only), fixed seed.\n\nRequires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Generated from [`pythondemos/transferlearning.py`](https://dictionaryofml.org/terms/transferlearning.py); CC BY 4.0."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Notebook shim: the script resolves output paths relative to __file__,\n# which a notebook kernel does not define; everything lands in the\n# working directory instead.\nimport os\n__file__ = os.path.join(os.getcwd(), \"transferlearning.py\")\nos.makedirs(\"pythondemos\", exist_ok=True)"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "\"\"\"\ntransferlearning.py \u2014 numerical companion to the glossary entry\n'transfer learning'.\n\nPurpose\n-------\nChecks the entry's claims on a small deep net with one-dimensional\ninput: a hypothesis learned on a source task with many data points,\nwhose early layer is kept and whose remaining model parameters are\nadapted by fine-tuning on a few data points of a target task, reaches a\nsmaller risk on the target task than a hypothesis learned from those few\ndata points alone; and when the features of the source task are not\nuseful for the target task, the fine-tuned hypothesis has a larger risk\nthan one learned from scratch (negative transfer).  Self-contained\n(numpy and matplotlib only), fixed seed.\n\nSetup\n-----\nDeep net with two layers: an early layer of 20 units computing the\nfeatures tanh(a_j x + b_j), and an output layer combining them\nlinearly.  Model parameters are learned by repeatedly adjusting them\nalong the direction that lowers the average squared error on the\ntraining set.  Source task: 400 data points with labels\nsin(3x) + 0.3 x + noise on [-2, 2].  Related target task: the same label\nfunction shifted up by 1.5, 8 training data points.  Unrelated source\ntask: a straight line 0.4 x + noise, whose learned features are nearly\nstraight lines too; its target task has labels sin(3x) + noise and 40\ntraining data points.  The risk on a target task is estimated on 2000\nfurther data points of that task.\n\nBlocks\n------\n[B-source]   The hypothesis learned on the source task reaches a small\n             risk on the source task (below 0.05).\n[B-transfer] Related target task, 8 data points: keeping the early layer\n             and fine-tuning the output layer gives a risk below 0.1,\n             less than a third of the risk of a hypothesis learned from\n             the 8 data points alone.\n[B-negative] Unrelated source task, 40 target data points: the\n             fine-tuned hypothesis keeps the straight-line features of\n             its source and has a risk more than twice that of the\n             hypothesis learned from scratch.\n\nOutputs\n-------\ntransferlearning_curves.csv : x grid with the related target label\n                              function, the source hypothesis, the\n                              fine-tuned hypothesis and the hypothesis\n                              learned from scratch.\ntransferlearning_target.csv : the 8 training data points of the related\n                              target task.\ntransferlearning.png        : matplotlib preview (checking only).\n\"\"\"\n\nimport numpy as np\nimport matplotlib\n\nmatplotlib.use(\"Agg\")\nimport matplotlib.pyplot as plt\n\nfrom pathlib import Path\n\nOUT_DIR = Path(__file__).parent\n\nreport = []\n\n\ndef check(name, ok):\n    report.append((name, bool(ok)))\n    print(f\"  [{'ok' if ok else 'FAIL'}] {name}\")\n\n\nrng = np.random.default_rng(0)\nJ = 20\n\n\ndef init():\n    return {\"a\": rng.standard_normal(J) * 2.0, \"b\": rng.uniform(-2, 2, J),\n            \"v\": rng.standard_normal(J) * 0.1, \"c\": 0.0}\n\n\ndef features(p, x):\n    return np.tanh(np.outer(x, p[\"a\"]) + p[\"b\"])      # early layer\n\n\ndef predict(p, x):\n    return features(p, x) @ p[\"v\"] + p[\"c\"]          # output layer\n\n\ndef train(p, x, y, steps, rate, early_layer=True):\n    \"\"\"Lower the average squared error on (x, y) by `steps` adjustments of\n    the model parameters; the early layer is kept fixed when\n    early_layer is False (fine-tuning of the output layer only).\"\"\"\n    p = {k: (np.array(v, dtype=float) if k != \"c\" else float(v)) for k, v in p.items()}\n    for _ in range(steps):\n        F = features(p, x); r = predict(p, x) - y; n = len(x)\n        gv = 2.0 / n * F.T @ r; gc = 2.0 / n * r.sum()\n        if early_layer:\n            dF = (1.0 - F ** 2) * np.outer(r, p[\"v\"]) * 2.0 / n\n            p[\"a\"] -= rate * (dF * x[:, None]).sum(axis=0)\n            p[\"b\"] -= rate * dF.sum(axis=0)\n        p[\"v\"] -= rate * gv; p[\"c\"] -= rate * gc\n    return p\n\n\ndef risk(p, fn, lo=-2.0, hi=2.0, n=2000):\n    xs = np.linspace(lo, hi, n)\n    return float(np.mean((predict(p, xs) - fn(xs)) ** 2))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-source]** The hypothesis learned on the source task reaches a small risk on the source task (below 0.05)."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "def f_source(x):\n    return np.sin(3.0 * x) + 0.3 * x\n\n\nx_s = rng.uniform(-2.0, 2.0, 400); y_s = f_source(x_s) + 0.1 * rng.standard_normal(400)\np_source = train(init(), x_s, y_s, steps=6000, rate=0.05)\nr_source = risk(p_source, f_source)\ncheck(f\"[B-source]   risk of the source hypothesis on the source task \"\n      f\"{r_source:.3f}\", r_source < 0.05)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-transfer]** Related target task, 8 data points: keeping the early layer and fine-tuning the output layer gives a risk below 0.1, less than a third of the risk of a hypothesis learned from the 8 data points alone."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "def f_target(x):\n    return f_source(x) + 1.5\n\n\nx_t = rng.uniform(-2.0, 2.0, 8); y_t = f_target(x_t) + 0.1 * rng.standard_normal(8)\np_tuned = train(p_source, x_t, y_t, steps=3000, rate=0.05, early_layer=False)\np_scratch = train(init(), x_t, y_t, steps=6000, rate=0.05)\nr_tuned, r_scratch = risk(p_tuned, f_target), risk(p_scratch, f_target)\ncheck(f\"[B-transfer] related target task, 8 data points: risk {r_tuned:.3f} \"\n      f\"fine-tuned vs {r_scratch:.3f} from scratch\",\n      r_tuned < 0.1 and r_tuned < r_scratch / 3)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-negative]** Unrelated source task, 40 target data points: the fine-tuned hypothesis keeps the straight-line features of its source and has a risk more than twice that of the hypothesis learned from scratch."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "def f_line(x):\n    return 0.4 * x\n\n\ndef f_unrel(x):\n    return np.sin(3.0 * x)\n\n\ny_line = f_line(x_s) + 0.1 * rng.standard_normal(400)\np_line = train(init(), x_s, y_line, steps=6000, rate=0.05)\nx_u = rng.uniform(-2.0, 2.0, 40); y_u = f_unrel(x_u) + 0.1 * rng.standard_normal(40)\np_neg = train(p_line, x_u, y_u, steps=3000, rate=0.05, early_layer=False)\np_neg_scratch = train(init(), x_u, y_u, steps=6000, rate=0.05)\nr_neg, r_neg_scratch = risk(p_neg, f_unrel), risk(p_neg_scratch, f_unrel)\ncheck(f\"[B-negative] unrelated source, 40 target data points: risk \"\n      f\"{r_neg:.3f} fine-tuned vs {r_neg_scratch:.3f} from scratch\",\n      r_neg > 2 * r_neg_scratch)\n\n# ---------------------------------------------------------------- CSV\nxg = np.linspace(-2.0, 2.0, 161)\nwith open(OUT_DIR / \"transferlearning_curves.csv\", \"w\") as fh:\n    fh.write(\"x,target,h_source,h_tuned,h_scratch\\n\")\n    for a, b, c, d, e in zip(xg, f_target(xg), predict(p_source, xg),\n                             predict(p_tuned, xg), predict(p_scratch, xg)):\n        fh.write(f\"{a:.4f},{b:.4f},{c:.4f},{d:.4f},{e:.4f}\\n\")\nwith open(OUT_DIR / \"transferlearning_target.csv\", \"w\") as fh:\n    fh.write(\"x,y\\n\")\n    for a, b in zip(x_t, y_t):\n        fh.write(f\"{a:.4f},{b:.4f}\\n\")\n\n# -------------------------------------------------------------- preview\nfig, (ax1, ax2) = plt.subplots(1, 2, figsize=(9.6, 3.9))\nax1.plot(x_s, y_s, \".\", color=\"0.75\", ms=3, label=\"source training set\")\nax1.plot(x_t, y_t, \"k^\", mfc=\"none\", ms=7, label=\"target training set (8 points)\")\nax1.plot(xg, predict(p_source, xg), \"k-\", lw=1.2, label=\"source hypothesis\")\nax1.plot(xg, predict(p_tuned, xg), \"k--\", lw=1.4, label=\"fine-tuned on target\")\nax1.plot(xg, predict(p_scratch, xg), \"k:\", lw=1.4, label=\"learned from target alone\")\nax1.set_xlabel(\"feature $x$\"); ax1.set_ylabel(\"label $y$\")\nax1.set_title(\"related target task: transfer helps\")\nax1.legend(frameon=False, fontsize=7)\nax2.plot(x_u, y_u, \"k^\", mfc=\"none\", ms=6, label=\"target training set (40 points)\")\nax2.plot(xg, f_unrel(xg), \"-\", color=\"0.6\", lw=1.0, label=\"target label function\")\nax2.plot(xg, predict(p_neg, xg), \"k--\", lw=1.4, label=\"fine-tuned from straight-line source\")\nax2.plot(xg, predict(p_neg_scratch, xg), \"k:\", lw=1.4, label=\"learned from scratch\")\nax2.set_xlabel(\"feature $x$\"); ax2.set_ylabel(\"label $y$\")\nax2.set_title(\"unrelated source task: negative transfer\")\nax2.legend(frameon=False, fontsize=7)\nfig.tight_layout()\nfig.savefig(OUT_DIR / \"transferlearning.png\", dpi=110)\n\nn_ok = sum(ok for _, ok in report)\nprint(f\"\\n{n_ok}/{len(report)} checks pass\")\nprint(f\"wrote {OUT_DIR / 'transferlearning_curves.csv'}, \"\n      f\"{OUT_DIR / 'transferlearning_target.csv'}, {OUT_DIR / 'transferlearning.png'}\")\nif n_ok != len(report):\n    raise SystemExit(1)"
  }
 ]
}