B-negative writes when the script runsDictionary of Applied Machine Learning · transfer learning
Numerical companion to the entry transfer learning: it recomputes what the entry states and prints one line per check
Checks the entry's claims on a small deep net with one-dimensional input: a hypothesis learned on a source task with many data points, whose early layer is kept and whose remaining model parameters are adapted by fine-tuning on a few data points of a target task, reaches a smaller risk on the target task than a hypothesis learned from those few data points alone; and when the features of the source task are not useful for the target task, the fine-tuned hypothesis has a larger risk than one learned from scratch (negative transfer). Self-contained (numpy and matplotlib only), fixed seed.
Run it without installing anything:uv run https://dictionaryofml.org/terms/transferlearning.py
uv downloads this script and the pinned NumPy and Matplotlib it needs, then runs it; the script fetches any input file it uses. To keep the output files, download transferlearning.py into a folder and run uv run transferlearning.py there. With NumPy and Matplotlib already installed, python3 transferlearning.py, from any directory — it writes its output files into the current directory. Fixed seeds, so the printed numbers reproduce exactly. Download transferlearning.py · Notebook · Open in Colab
One cell per block of the script: the code, and what that code printed when it last ran here
"""
transferlearning.py — numerical companion to the glossary entry
'transfer learning'.
Purpose
-------
Checks the entry's claims on a small deep net with one-dimensional
input: a hypothesis learned on a source task with many data points,
whose early layer is kept and whose remaining model parameters are
adapted by fine-tuning on a few data points of a target task, reaches a
smaller risk on the target task than a hypothesis learned from those few
data points alone; and when the features of the source task are not
useful for the target task, the fine-tuned hypothesis has a larger risk
than one learned from scratch (negative transfer). Self-contained
(numpy and matplotlib only), fixed seed.
Setup
-----
Deep net with two layers: an early layer of 20 units computing the
features tanh(a_j x + b_j), and an output layer combining them
linearly. Model parameters are learned by repeatedly adjusting them
along the direction that lowers the average squared error on the
training set. Source task: 400 data points with labels
sin(3x) + 0.3 x + noise on [-2, 2]. Related target task: the same label
function shifted up by 1.5, 8 training data points. Unrelated source
task: a straight line 0.4 x + noise, whose learned features are nearly
straight lines too; its target task has labels sin(3x) + noise and 40
training data points. The risk on a target task is estimated on 2000
further data points of that task.
Blocks
------
[B-source] The hypothesis learned on the source task reaches a small
risk on the source task (below 0.05).
[B-transfer] Related target task, 8 data points: keeping the early layer
and fine-tuning the output layer gives a risk below 0.1,
less than a third of the risk of a hypothesis learned from
the 8 data points alone.
[B-negative] Unrelated source task, 40 target data points: the
fine-tuned hypothesis keeps the straight-line features of
its source and has a risk more than twice that of the
hypothesis learned from scratch.
Outputs
-------
transferlearning_curves.csv : x grid with the related target label
function, the source hypothesis, the
fine-tuned hypothesis and the hypothesis
learned from scratch.
transferlearning_target.csv : the 8 training data points of the related
target task.
transferlearning.png : matplotlib preview (checking only).
"""
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from pathlib import Path
OUT_DIR = Path(__file__).parent
report = []
def check(name, ok):
report.append((name, bool(ok)))
print(f" [{'ok' if ok else 'FAIL'}] {name}")
rng = np.random.default_rng(0)
J = 20
def init():
return {"a": rng.standard_normal(J) * 2.0, "b": rng.uniform(-2, 2, J),
"v": rng.standard_normal(J) * 0.1, "c": 0.0}
def features(p, x):
return np.tanh(np.outer(x, p["a"]) + p["b"]) # early layer
def predict(p, x):
return features(p, x) @ p["v"] + p["c"] # output layer
def train(p, x, y, steps, rate, early_layer=True):
"""Lower the average squared error on (x, y) by `steps` adjustments of
the model parameters; the early layer is kept fixed when
early_layer is False (fine-tuning of the output layer only)."""
p = {k: (np.array(v, dtype=float) if k != "c" else float(v)) for k, v in p.items()}
for _ in range(steps):
F = features(p, x); r = predict(p, x) - y; n = len(x)
gv = 2.0 / n * F.T @ r; gc = 2.0 / n * r.sum()
if early_layer:
dF = (1.0 - F ** 2) * np.outer(r, p["v"]) * 2.0 / n
p["a"] -= rate * (dF * x[:, None]).sum(axis=0)
p["b"] -= rate * dF.sum(axis=0)
p["v"] -= rate * gv; p["c"] -= rate * gc
return p
def risk(p, fn, lo=-2.0, hi=2.0, n=2000):
xs = np.linspace(lo, hi, n)
return float(np.mean((predict(p, xs) - fn(xs)) ** 2))
The hypothesis learned on the source task reaches a small risk on the source task (below 0.05).
def f_source(x):
return np.sin(3.0 * x) + 0.3 * x
x_s = rng.uniform(-2.0, 2.0, 400); y_s = f_source(x_s) + 0.1 * rng.standard_normal(400)
p_source = train(init(), x_s, y_s, steps=6000, rate=0.05)
r_source = risk(p_source, f_source)
check(f"[B-source] risk of the source hypothesis on the source task "
f"{r_source:.3f}", r_source < 0.05)
[ok] [B-source] risk of the source hypothesis on the source task 0.003
Related target task, 8 data points: keeping the early layer and fine-tuning the output layer gives a risk below 0.1, less than a third of the risk of a hypothesis learned from the 8 data points alone.
def f_target(x):
return f_source(x) + 1.5
x_t = rng.uniform(-2.0, 2.0, 8); y_t = f_target(x_t) + 0.1 * rng.standard_normal(8)
p_tuned = train(p_source, x_t, y_t, steps=3000, rate=0.05, early_layer=False)
p_scratch = train(init(), x_t, y_t, steps=6000, rate=0.05)
r_tuned, r_scratch = risk(p_tuned, f_target), risk(p_scratch, f_target)
check(f"[B-transfer] related target task, 8 data points: risk {r_tuned:.3f} "
f"fine-tuned vs {r_scratch:.3f} from scratch",
r_tuned < 0.1 and r_tuned < r_scratch / 3)
[ok] [B-transfer] related target task, 8 data points: risk 0.010 fine-tuned vs 0.097 from scratch
Unrelated source task, 40 target data points: the fine-tuned hypothesis keeps the straight-line features of its source and has a risk more than twice that of the hypothesis learned from scratch.
def f_line(x):
return 0.4 * x
def f_unrel(x):
return np.sin(3.0 * x)
y_line = f_line(x_s) + 0.1 * rng.standard_normal(400)
p_line = train(init(), x_s, y_line, steps=6000, rate=0.05)
x_u = rng.uniform(-2.0, 2.0, 40); y_u = f_unrel(x_u) + 0.1 * rng.standard_normal(40)
p_neg = train(p_line, x_u, y_u, steps=3000, rate=0.05, early_layer=False)
p_neg_scratch = train(init(), x_u, y_u, steps=6000, rate=0.05)
r_neg, r_neg_scratch = risk(p_neg, f_unrel), risk(p_neg_scratch, f_unrel)
check(f"[B-negative] unrelated source, 40 target data points: risk "
f"{r_neg:.3f} fine-tuned vs {r_neg_scratch:.3f} from scratch",
r_neg > 2 * r_neg_scratch)
# ---------------------------------------------------------------- CSV
xg = np.linspace(-2.0, 2.0, 161)
with open(OUT_DIR / "transferlearning_curves.csv", "w") as fh:
fh.write("x,target,h_source,h_tuned,h_scratch\n")
for a, b, c, d, e in zip(xg, f_target(xg), predict(p_source, xg),
predict(p_tuned, xg), predict(p_scratch, xg)):
fh.write(f"{a:.4f},{b:.4f},{c:.4f},{d:.4f},{e:.4f}\n")
with open(OUT_DIR / "transferlearning_target.csv", "w") as fh:
fh.write("x,y\n")
for a, b in zip(x_t, y_t):
fh.write(f"{a:.4f},{b:.4f}\n")
# -------------------------------------------------------------- preview
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(9.6, 3.9))
ax1.plot(x_s, y_s, ".", color="0.75", ms=3, label="source training set")
ax1.plot(x_t, y_t, "k^", mfc="none", ms=7, label="target training set (8 points)")
ax1.plot(xg, predict(p_source, xg), "k-", lw=1.2, label="source hypothesis")
ax1.plot(xg, predict(p_tuned, xg), "k--", lw=1.4, label="fine-tuned on target")
ax1.plot(xg, predict(p_scratch, xg), "k:", lw=1.4, label="learned from target alone")
ax1.set_xlabel("feature $x$"); ax1.set_ylabel("label $y$")
ax1.set_title("related target task: transfer helps")
ax1.legend(frameon=False, fontsize=7)
ax2.plot(x_u, y_u, "k^", mfc="none", ms=6, label="target training set (40 points)")
ax2.plot(xg, f_unrel(xg), "-", color="0.6", lw=1.0, label="target label function")
ax2.plot(xg, predict(p_neg, xg), "k--", lw=1.4, label="fine-tuned from straight-line source")
ax2.plot(xg, predict(p_neg_scratch, xg), "k:", lw=1.4, label="learned from scratch")
ax2.set_xlabel("feature $x$"); ax2.set_ylabel("label $y$")
ax2.set_title("unrelated source task: negative transfer")
ax2.legend(frameon=False, fontsize=7)
fig.tight_layout()
fig.savefig(OUT_DIR / "transferlearning.png", dpi=110)
n_ok = sum(ok for _, ok in report)
print(f"\n{n_ok}/{len(report)} checks pass")
print(f"wrote {OUT_DIR / 'transferlearning_curves.csv'}, "
f"{OUT_DIR / 'transferlearning_target.csv'}, {OUT_DIR / 'transferlearning.png'}")
if n_ok != len(report):
raise SystemExit(1)
[ok] [B-negative] unrelated source, 40 target data points: risk 0.041 fine-tuned vs 0.008 from scratch 3/3 checks pass wrote /Users/junga1/dictionaryappliedml/pythondemos/transferlearning_curves.csv, /Users/junga1/dictionaryappliedml/pythondemos/transferlearning_target.csv, /Users/junga1/dictionaryappliedml/pythondemos/transferlearning.png
B-negative writes when the script runs