B-picture writes when the script runsDictionary of Applied Machine Learning · training set
Numerical companion to the entry training set: it recomputes what the entry states and prints one line per check
The entry's weather narrative, carried out: three days of synthetic weather recordings (morning minimum temperature as feature, maximum daytime temperature as label) form the training set, and two hypotheses are learned from it -- a straight line and a degree-two polynomial. Every claim of the entry is measured on the result. Self-contained (numpy/matplotlib only), deterministic.
Run it with python3 trainset.py, from any directory — it writes its output files into the current directory. Requires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Download trainset.py · Notebook · Open in Colab
One cell per block of the script: the code, and what that code printed when it last ran here
"""
trainset.py -- numerical companion to the entry 'training set'.
The entry's weather narrative, carried out: three days of synthetic weather
recordings (morning minimum temperature as feature, maximum daytime
temperature as label) form the training set, and two hypotheses are learned
from it -- a straight line and a degree-two polynomial. Every claim of the
entry is measured on the result. Self-contained (numpy/matplotlib only),
deterministic.
Blocks
------
[B-trainset] Three days of recordings form the training set: for each day
the morning minimum temperature (feature) and the maximum
daytime temperature (label) are known, so the loss of any
candidate hypothesis can be evaluated on them.
[B-erm] Fit a straight line by minimizing the average squared error
over the training set. The value of that minimum is the
training error, and any other line incurs a larger average.
[B-poly] Fit a degree-two polynomial to the same training set: it
passes through all three data points, so its training error
is zero -- smaller than the training error of the line.
[B-misleading] A small training error can be misleading: on twenty held-back
validation days, the polynomial's average loss (its validation
error) far exceeds the line's, reversing the training-set
ranking (overfitting).
[B-picture] The training set and both hypotheses evaluated on a grid,
written to CSV for the entry's figure.
Outputs
-------
pythondemos/trainset.png : preview figure (checking only).
pythondemos/trainset_train.csv : the three training days (x = morning
minimum temperature, y = maximum daytime
temperature)
pythondemos/trainset_line.csv : the fitted line on a two-point grid
pythondemos/trainset_poly.csv : the fitted polynomial on a dense grid
"""
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from pathlib import Path
OUT_DIR = Path(__file__).parent
report = []
def check(name, ok):
report.append((name, bool(ok)))
print(f" [{'ok' if ok else 'FAIL'}] {name}")
def days(n, gen, lo=-15.0, hi=5.0, w1=0.8, w0=4.0, noise=2.0):
"""n days: morning minimum x and maximum daytime temperature y."""
x = np.sort(gen.uniform(lo, hi, n))
y = w0 + w1 * x + gen.normal(0.0, noise, n)
return x, y
def avg_sqerr(x, y, coeffs):
return float(np.mean((y - np.polyval(coeffs, x)) ** 2))
Three days of recordings form the training set: for each day the morning minimum temperature (feature) and the maximum daytime temperature (label) are known, so the loss of any candidate hypothesis can be evaluated on them.
print("[B-trainset] three days of recordings form the training set")
x, y = days(3, np.random.default_rng(100))
for xr, yr in zip(x, y):
print(f" morning minimum {xr:6.1f} C, maximum daytime {yr:6.1f} C")
check("both temperatures of every training day are known",
np.all(np.isfinite(x)) and np.all(np.isfinite(y)))
[B-trainset] three days of recordings form the training set
morning minimum -9.2 C, maximum daytime -2.3 C
morning minimum -3.1 C, maximum daytime -0.4 C
morning minimum 1.7 C, maximum daytime 7.5 C
[ok] both temperatures of every training day are known
Fit a straight line by minimizing the average squared error over the training set. The value of that minimum is the training error, and any other line incurs a larger average.
print("\n[B-erm] the line minimizes the average squared error over the "
"training set")
c_line = np.polyfit(x, y, 1)
trainerr_line = avg_sqerr(x, y, c_line)
print(f" fitted line: max temperature = {c_line[1]:.2f} + {c_line[0]:.2f}"
f" * min temperature; training error {trainerr_line:.3f}")
gen = np.random.default_rng(7)
others = [avg_sqerr(x, y, c_line + d) for d in gen.normal(0.0, 0.3, (200, 2))]
check("every perturbed line incurs a larger average on the training set",
all(o >= trainerr_line for o in others))
[B-erm] the line minimizes the average squared error over the training set
fitted line: max temperature = 4.68 + 0.87 * min temperature; training error 2.872
[ok] every perturbed line incurs a larger average on the training set
Fit a degree-two polynomial to the same training set: it passes through all three data points, so its training error is zero -- smaller than the training error of the line.
print("\n[B-poly] a degree-two polynomial passes through all three days")
c_poly = np.polyfit(x, y, 2)
trainerr_poly = avg_sqerr(x, y, c_poly)
print(f" polynomial training error {trainerr_poly:.2e} "
f"(line: {trainerr_line:.3f})")
check("the polynomial's training error is zero (up to round-off)",
trainerr_poly < 1e-10)
check("the polynomial's training error is smaller than the line's",
trainerr_poly < trainerr_line)
[B-poly] a degree-two polynomial passes through all three days
polynomial training error 1.11e-29 (line: 2.872)
[ok] the polynomial's training error is zero (up to round-off)
[ok] the polynomial's training error is smaller than the line's
A small training error can be misleading: on twenty held-back validation days, the polynomial's average loss (its validation error) far exceeds the line's, reversing the training-set ranking (overfitting).
print("\n[B-misleading] the training-set ranking of the two hypotheses is "
"misleading")
xv, yv = days(20, np.random.default_rng(1100))
valerr_line = avg_sqerr(xv, yv, c_line)
valerr_poly = avg_sqerr(xv, yv, c_poly)
print(f" validation error on 20 held-back days: line {valerr_line:.3f}, "
f"polynomial {valerr_poly:.3f}")
check("the polynomial's validation error far exceeds the line's",
valerr_poly > 3 * valerr_line)
check("the validation days are disjoint from the training days",
len(set(np.round(xv, 6)) & set(np.round(x, 6))) == 0)
[B-misleading] the training-set ranking of the two hypotheses is misleading
validation error on 20 held-back days: line 3.167, polynomial 26.175
[ok] the polynomial's validation error far exceeds the line's
[ok] the validation days are disjoint from the training days
The training set and both hypotheses evaluated on a grid, written to CSV for the entry's figure.
print("\n[B-picture] training set and both hypotheses, written to CSV")
xs_line = np.array([-14.0, 5.0])
xs_poly = np.linspace(-14.0, 5.0, 200)
def write_csv(name, cols, header):
np.savetxt(OUT_DIR / name, np.column_stack(cols), delimiter=",",
header=header, comments="", fmt="%.4f")
write_csv("trainset_train.csv", (x, y), "x,y")
write_csv("trainset_line.csv", (xs_line, np.polyval(c_line, xs_line)),
"x,yhat")
write_csv("trainset_poly.csv", (xs_poly, np.polyval(c_poly, xs_poly)),
"x,yhat")
print(f" wrote {len(x)} training days, the line and the polynomial to "
f"3 CSV files")
check("the two curves agree on the training days only (max gap elsewhere "
"is large)",
np.max(np.abs(np.polyval(c_poly, xs_poly)
- np.polyval(c_line, xs_poly))) > 3.0)
fig, ax = plt.subplots(figsize=(5.6, 3.6))
ax.plot(xs_poly, np.polyval(c_poly, xs_poly), "--", color="black", lw=1.2,
label="polynomial (training error zero)")
ax.plot(xs_line, np.polyval(c_line, xs_line), "-", color="black", lw=1.2,
label="straight line")
ax.plot(x, y, "o", ms=6, color="black", label="training set")
ax.set_xlabel("morning minimum temperature")
ax.set_ylabel("maximum daytime temperature")
ax.set_title("three training days, two hypotheses learned from them",
fontsize=9)
ax.legend(frameon=False, fontsize=8)
fig.tight_layout()
fig.savefig(OUT_DIR / "trainset.png", dpi=110)
n_ok = sum(ok for _, ok in report)
print(f"\n{n_ok}/{len(report)} checks pass")
print("wrote trainset.png")
if n_ok != len(report):
raise SystemExit(1)
[B-picture] training set and both hypotheses, written to CSV
wrote 3 training days, the line and the polynomial to 3 CSV files
[ok] the two curves agree on the training days only (max gap elsewhere is large)
7/7 checks pass
wrote trainset.png
B-picture writes when the script runs