P-noise writes when the script runsDictionary of Applied Machine Learning · label
Numerical companion to the entry label: it recomputes what the entry states and prints one line per check
One block per paragraph of the entry (marked [P...]): each block verifies numerically what the corresponding statement asserts. Self-contained (numpy/matplotlib only), fixed seed.
Run it without installing anything:uv run https://dictionaryofml.org/terms/label.py
uv downloads this script and the pinned NumPy and Matplotlib it needs, then runs it; the script fetches any input file it uses. To keep the output files, download label.py into a folder and run uv run label.py there. With NumPy and Matplotlib already installed, python3 label.py, from any directory — it writes its output files into the current directory. Fixed seeds, so the printed numbers reproduce exactly. Download label.py · Notebook · Open in Colab
One cell per block of the script: the code, and what that code printed when it last ran here
"""
label.py — numerical companion to the glossary entry 'label'.
One block per paragraph of the entry (marked [P...]): each block verifies
numerically what the corresponding statement asserts. Self-contained
(numpy/matplotlib only), fixed seed.
Blocks
------
[P-choice] Feature vs label is a design choice: with the "oncologist
available" design the prognosis enters the feature vector
and improves prediction of the target; without it the
prognosis is the label to be predicted from the remaining
features.
[P-labelfun] The labeling function h-bar acts on the data point itself:
a deterministic function assigns each (fully observed)
synthetic patient its label.
[P-partial] The feature vector captures only part of the data point:
two data points with identical features carry different
labels, so no hypothesis reading only the features predicts
perfectly — the empirical error of the best such hypothesis
stays bounded away from zero, while a hypothesis with
access to the full data point achieves zero error.
[P-noise] The observed label is a noisy proxy: flipping a growing
fraction of the training labels degrades the test error
of the learned hypothesis (label noise degrades training
and generalization).
Outputs
-------
label.png : preview figure (checking only).
Data generated by pythondemos/label.py.
"""
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from pathlib import Path
OUT_DIR = Path(__file__).parent
rng = np.random.default_rng(42)
report = []
def check(name, ok):
report.append((name, bool(ok)))
print(f" [{'ok' if ok else 'FAIL'}] {name}")
# synthetic patients: (weight, age, hidden_marker) fully describe the
# data point; the label depends on all three.
m = 600
weight = 60 + 20 * rng.random(m)
age = 20 + 50 * rng.random(m)
hidden = rng.normal(size=m) # never recorded as feature
z = np.stack([weight, age, hidden], axis=1) # the data point itself
label_fun = lambda Z: (0.03 * Z[:, 0] + 0.04 * Z[:, 1]
+ 1.5 * Z[:, 2] > 4.0) # labeling function h-bar
y = label_fun(z)
Feature vs label is a design choice: with the "oncologist available" design the prognosis enters the feature vector and improves prediction of the target; without it the prognosis is the label to be predicted from the remaining features.
print("[P-choice] feature vs label as a design choice")
prognosis = (0.5 * hidden + 0.2 * rng.normal(size=m) > 0) # physician's call
target = y
def acc(X, yy):
Xb = np.c_[X, np.ones(m)]
w = np.linalg.lstsq(Xb, yy.astype(float), rcond=None)[0]
return np.mean((Xb @ w > 0.5) == yy)
acc_without = acc(np.c_[weight, age], target)
acc_with = acc(np.c_[weight, age, prognosis.astype(float)], target)
print(f" accuracy without / with the prognosis feature: "
f"{acc_without:.2f} / {acc_with:.2f}")
check("prognosis used as a FEATURE improves the prediction",
acc_with > acc_without + 0.05)
check("prognosis used as the LABEL is predictable from features "
"(better than chance)",
acc(np.c_[weight, age], prognosis) >= 0.5)
[P-choice] feature vs label as a design choice
accuracy without / with the prognosis feature: 0.64 / 0.84
[ok] prognosis used as a FEATURE improves the prediction
[ok] prognosis used as the LABEL is predictable from features (better than chance)
The labeling function h-bar acts on the data point itself: a deterministic function assigns each (fully observed) synthetic patient its label.
print("[P-labelfun] the labeling function acts on the data point")
check("h-bar is deterministic on data points",
np.array_equal(label_fun(z), y))
check("h-bar reads the whole data point (hidden part matters)",
not np.array_equal(label_fun(z),
label_fun(np.c_[z[:, :2], np.zeros(m)])))
[P-labelfun] the labeling function acts on the data point [ok] h-bar is deterministic on data points [ok] h-bar reads the whole data point (hidden part matters)
The feature vector captures only part of the data point: two data points with identical features carry different labels, so no hypothesis reading only the features predicts perfectly — the empirical error of the best such hypothesis stays bounded away from zero, while a hypothesis with access to the full data point achieves zero error.
print("[P-partial] identical features, different labels")
x_feat = np.round(np.stack([weight, age], axis=1)) # recorded features
# find twin data points with equal features but different labels
_, inv, counts = np.unique(x_feat, axis=0, return_inverse=True,
return_counts=True)
twin_exists = any(len(set(y[inv == g])) > 1
for g in np.nonzero(counts > 1)[0])
check("two data points share features but differ in the label",
twin_exists)
err_feat = 1 - acc(x_feat, y)
Xfull = np.c_[z, np.ones(m)]
w_full = np.linalg.lstsq(Xfull, y.astype(float), rcond=None)[0]
err_full = np.mean((Xfull @ w_full > 0.5) != y)
print(f" error with features only: {err_feat:.2f}; "
f"with the full data point: {err_full:.2f}")
check("features-only hypothesis cannot be perfect", err_feat > 0.05)
check("full-data-point hypothesis is (nearly) perfect", err_full < 0.02)
[P-partial] identical features, different labels
[ok] two data points share features but differ in the label
error with features only: 0.36; with the full data point: 0.01
[ok] features-only hypothesis cannot be perfect
[ok] full-data-point hypothesis is (nearly) perfect
The observed label is a noisy proxy: flipping a growing fraction of the training labels degrades the test error of the learned hypothesis (label noise degrades training and generalization).
print("[P-noise] label noise degrades training and generalization")
Xtr, Xte = x_feat[:400], x_feat[400:]
ytr0, yte = y[:400], y[400:]
errs = []
for flip in (0.0, 0.2, 0.4):
errs_r = []
for _ in range(25):
ytr = ytr0.copy()
idx = rng.choice(400, int(flip * 400), replace=False)
ytr[idx] = ~ytr[idx]
Xb = np.c_[Xtr, np.ones(400)]
w = np.linalg.lstsq(Xb, ytr.astype(float), rcond=None)[0]
errs_r.append(np.mean((np.c_[Xte, np.ones(200)] @ w > 0.5)
!= yte))
errs.append(float(np.mean(errs_r)))
print(f" test error at flip = 0, 0.2, 0.4: "
f"{errs[0]:.2f}, {errs[1]:.2f}, {errs[2]:.2f}")
check("test error grows with the label-noise level",
errs[0] < errs[1] < errs[2])
# ------------------------------------------------------------ preview
fig, ax = plt.subplots(figsize=(4.6, 3.2))
ax.plot([0, 0.2, 0.4], errs, "o-")
ax.set_xlabel("fraction of flipped labels"); ax.set_ylabel("test error")
ax.set_title("[P-noise] label noise degrades generalization")
fig.tight_layout()
fig.savefig(OUT_DIR / "label.png", dpi=110)
print(f"\n{sum(ok for _, ok in report)}/{len(report)} checks passed")
assert all(ok for _, ok in report)
[P-noise] label noise degrades training and generalization
test error at flip = 0, 0.2, 0.4: 0.34, 0.34, 0.40
[ok] test error grows with the label-noise level
8/8 checks passed
P-noise writes when the script runs