B-fig writes when the script runsDictionary of Applied Machine Learning · unsupervised learning
Updated on 2026-11-03
Numerical companion to the entry unsupervised learning: it recomputes what the entry states and prints one line per check
Numerical companion to the glossary entry 'unsupervisedlearning'. The GeoSphere Austria weather station Krems (station id 3805) records a daily minimum and maximum air temperature. This script downloads the records for 2024 from the GeoSphere data hub (dataset klima-v2-1d), writes them to unsupervisedlearning_weather.csv, and keeps the first ten days of January and the first ten days of July. Each of those twenty days is a data point with two features and no label: nothing in the record says which days belong together or what should be predicted for a day.
Run it without installing anything:uv run https://dictionaryofml.org/terms/unsupervisedlearning.py
uv downloads this script and the pinned NumPy and Matplotlib it needs, then runs it; the script fetches any input file it uses. To keep the output files, download unsupervisedlearning.py into a folder and run uv run unsupervisedlearning.py there. With NumPy and Matplotlib already installed, python3 unsupervisedlearning.py, from any directory — it writes its output files into the current directory. Fixed seeds, so the printed numbers reproduce exactly. Download unsupervisedlearning.py · Notebook · Open in Colab
Before running a downloaded copy, scan it: uvx bandit -q -t B102,B307,B602,B605 unsupervisedlearning.py (B102,B307,B602,B605 are the exec, eval and shell checks of bandit) reports nothing: the script runs no shell command and evaluates no generated code.
One cell per block of the script: the code, and what that code printed when it last ran here
"""Unsupervised learning on twenty days at Krems: no label anywhere, and
model parameters still learned by ERM -- two cluster centroids, and a
linear feature transformation that turns two numbers into one.
Purpose
-------
Numerical companion to the glossary entry 'unsupervisedlearning'. The
GeoSphere Austria weather station Krems (station id 3805) records a
daily minimum and maximum air temperature. This script downloads the
records for 2024 from the GeoSphere data hub (dataset klima-v2-1d),
writes them to unsupervisedlearning_weather.csv, and keeps the first
ten days of January and the first ten days of July. Each of those
twenty days is a data point with two features and no label: nothing in
the record says which days belong together or what should be predicted
for a day.
Two methods are run on the same twenty unlabeled days, and both learn
their model parameters by minimizing an average loss over the dataset:
k-means, whose model parameters are two cluster centroids and whose
loss is the squared distance from a day to the nearer centroid;
a linear feature transformation onto one direction, whose parameter is
that direction and whose loss is the squared distance from a day to
its orthogonal projection on the line through the sample mean.
Neither loss can be evaluated for a single day on its own: both are
defined against model parameters that summarize the whole dataset.
Deterministic: the data are a fixed archive year, the initial centroids
are the coldest and the warmest of the twenty days, and the direction
is the leading direction of the sample covariance matrix. Self-contained:
numpy + matplotlib only (stdlib urllib for the download).
Blocks
------
[B-data] Download the 366 days of 2024 and keep the first ten days
of January and of July as twenty unlabeled data points
with two features, the daily minimum and maximum
temperature in degrees Celsius.
[B-cluster] k-means with k = 2: check the clustering error never
increases, that the iteration reaches a fixed point, and
that the two clusters recover the winter and the summer
days exactly -- a comparison made only after the run,
never during it.
[B-line] The linear feature transformation: the direction that
minimizes the squared projection error, taken as the leading
direction of the sample covariance matrix. Check that it beats every other
direction tried and that it keeps almost all the variance.
[B-fig] Preview: the twenty days, an ellipse around each cluster,
the line, and the orthogonal projection of two days on it.
Outputs
-------
unsupervisedlearning_weather.csv : date and the eight measurements
unsupervisedlearning_days.csv : tmin, tmax of the twenty days
unsupervisedlearning_centroids.csv: tmin, tmax of the two centroids
unsupervisedlearning_ellipse1.csv, _ellipse2.csv : one closed curve each
unsupervisedlearning_line.csv : the two endpoints of the line
unsupervisedlearning_proj.csv : the two projection segments
unsupervisedlearning_feet.csv : the two projected days
unsupervisedlearning.png : preview (checking only)
"""
# /// script
# requires-python = ">=3.10"
# dependencies = ["matplotlib==3.10.7", "numpy==2.3.4"]
# ///
import json
import urllib.request
from pathlib import Path
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
OUT_DIR = Path(__file__).parent
report = []
def check(name, ok):
report.append((name, bool(ok)))
print(f" [{'ok' if ok else 'FAIL'}] {name}")
Download the 366 days of 2024 and keep the first ten days of January and of July as twenty unlabeled data points with two features, the daily minimum and maximum temperature in degrees Celsius.
PARAMS = ["tlmin", "tlmax", "tl_mittel", "rr", "so_h", "rf_mittel",
"p_mittel", "vv_mittel"]
URL = ("https://dataset.api.hub.geosphere.at/v1/station/historical/"
f"klima-v2-1d?parameters={','.join(PARAMS)}&station_ids=3805"
"&start=2024-01-01&end=2024-12-31")
with urllib.request.urlopen(URL, timeout=120) as resp:
payload = json.load(resp)
params = payload["features"][0]["properties"]["parameters"]
stamps = [t[:10] for t in payload["timestamps"]]
M = np.stack([np.array(params[p]["data"], dtype=float) for p in PARAMS], 1)
M[:, 3] = np.maximum(M[:, 3], 0.0) # -1 marks a trace of rain
with open(OUT_DIR / "unsupervisedlearning_weather.csv", "w") as f:
f.write("date," + ",".join(PARAMS) + "\n")
for day, row in zip(stamps, M):
f.write(day + "," + ",".join(f"{v:g}" for v in row) + "\n")
winter = [i for i, s in enumerate(stamps) if s.startswith("2024-01")][:10]
summer = [i for i, s in enumerate(stamps) if s.startswith("2024-07")][:10]
X = M[winter + summer][:, :2] # tlmin, tlmax in degrees C
SEASON = np.array([0] * 10 + [1] * 10) # kept out of every fit below
m = len(X)
np.savetxt(OUT_DIR / "unsupervisedlearning_days.csv", X,
delimiter=",", header="tmin,tmax", comments="", fmt="%.2f")
check("[B-data] twenty days with two features each", X.shape == (20, 2))
check("[B-data] the features are temperatures in a plausible range",
-15.0 < X.min() and X.max() < 40.0)
print(f" twenty days at Krems: minimum {X[:, 0].min():.1f} to "
f"{X[:, 0].max():.1f}, maximum {X[:, 1].min():.1f} to "
f"{X[:, 1].max():.1f} degrees")
[ok] [B-data] twenty days with two features each [ok] [B-data] the features are temperatures in a plausible range twenty days at Krems: minimum -9.0 to 18.9, maximum -3.6 to 34.3 degrees
k-means with k = 2: check the clustering error never increases, that the iteration reaches a fixed point, and that the two clusters recover the winter and the summer days exactly -- a comparison made only after the run, never during it.
def kmeans(X, centroids, steps=50):
"""Lloyd's algorithm; returns the centroids, the assignment, the errors."""
errors = []
labels = np.zeros(len(X), dtype=int)
for _ in range(steps):
dist = ((X[:, None, :] - centroids[None, :, :]) ** 2).sum(axis=2)
labels = dist.argmin(axis=1)
errors.append(float(dist.min(axis=1).mean()))
moved = np.stack([X[labels == c].mean(axis=0)
for c in range(len(centroids))])
if np.allclose(moved, centroids):
centroids = moved
break
centroids = moved
return centroids, labels, errors
start = np.stack([X[X[:, 1].argmin()], X[X[:, 1].argmax()]])
centroids, labels, errors = kmeans(X, start.copy())
np.savetxt(OUT_DIR / "unsupervisedlearning_centroids.csv", centroids,
delimiter=",", header="tmin,tmax", comments="", fmt="%.2f")
check("[B-cluster] the average loss never increases",
all(b <= a + 1e-12 for a, b in zip(errors, errors[1:])))
check("[B-cluster] the iteration reaches a fixed point", len(errors) < 50)
agree = max((labels == SEASON).mean(), (labels != SEASON).mean())
check("[B-cluster] the two clusters recover the winter and summer days",
agree == 1.0)
print(f" average loss of the learned centroids: {errors[-1]:.2f} "
f"squared degrees, reached after {len(errors)} iterations")
# one ellipse per cluster, scaled to enclose its own days
ell = []
for c in (0, 1):
P = X[labels == c]
mu = P.mean(axis=0)
C = np.cov((P - mu).T) + 1e-9 * np.eye(2)
L = np.linalg.cholesky(C)
radii = np.linalg.solve(L, (P - mu).T)
scale = 1.08 * np.sqrt((radii ** 2).sum(axis=0)).max()
t = np.linspace(0.0, 2.0 * np.pi, 241)
curve = (mu[:, None] + scale * L @ np.stack([np.cos(t), np.sin(t)])).T
ell.append(curve)
np.savetxt(OUT_DIR / f"unsupervisedlearning_ellipse{c + 1}.csv", curve,
delimiter=",", header="tmin,tmax", comments="", fmt="%.3f")
inside = np.linalg.solve(L, (P - mu).T)
check(f"[B-cluster] ellipse {c + 1} encloses its own days",
bool(np.all(np.sqrt((inside ** 2).sum(axis=0)) <= scale)))
[ok] [B-cluster] the average loss never increases [ok] [B-cluster] the iteration reaches a fixed point [ok] [B-cluster] the two clusters recover the winter and summer days average loss of the learned centroids: 33.72 squared degrees, reached after 2 iterations [ok] [B-cluster] ellipse 1 encloses its own days [ok] [B-cluster] ellipse 2 encloses its own days
The linear feature transformation: the direction that minimizes the squared projection error, taken as the leading direction of the sample covariance matrix. Check that it beats every other direction tried and that it keeps almost all the variance.
mu = X.mean(axis=0)
C = np.cov((X - mu).T)
evals, evecs = np.linalg.eigh(C)
direction = evecs[:, -1]
proj_error = float((((X - mu) - np.outer((X - mu) @ direction,
direction)) ** 2).sum(axis=1).mean())
angles = np.linspace(0.0, np.pi, 721)
tried = np.stack([np.cos(angles), np.sin(angles)])
other = (((X - mu) ** 2).sum(axis=1).mean()
- (((X - mu) @ tried) ** 2).mean(axis=0))
check("[B-line] no other direction gives a smaller average loss",
proj_error <= other.min() + 1e-9)
kept = float(evals[-1] / evals.sum())
check("[B-line] the one learned feature keeps almost all the variance",
kept > 0.95)
print(f" the learned direction leaves an average loss of "
f"{proj_error:.2f} squared degrees and keeps {kept:.1%} "
f"of the total variance")
span = (X - mu) @ direction
ends = np.stack([mu + 1.12 * span.min() * direction,
mu + 1.12 * span.max() * direction])
np.savetxt(OUT_DIR / "unsupervisedlearning_line.csv", ends,
delimiter=",", header="tmin,tmax", comments="", fmt="%.3f")
# two days, one per cluster: the ones the line moves furthest, so that the
# right angle of the projection is visible rather than a few pixels long
off = np.abs((X - mu) @ np.array([-direction[1], direction[0]]))
pick = [int(np.where(labels == c)[0][off[labels == c].argmax()])
for c in (labels[0], 1 - labels[0])]
feet = mu + np.outer((X[pick] - mu) @ direction, direction)
np.savetxt(OUT_DIR / "unsupervisedlearning_feet.csv", feet,
delimiter=",", header="tmin,tmax", comments="", fmt="%.3f")
with open(OUT_DIR / "unsupervisedlearning_proj.csv", "w") as f:
f.write("tmin,tmax\n")
for p, q in zip(X[pick], feet):
f.write(f"{p[0]:.3f},{p[1]:.3f}\n{q[0]:.3f},{q[1]:.3f}\nnan,nan\n")
for p, q in zip(X[pick], feet):
check("[B-line] the projection segment meets the line at a right angle",
abs(float((p - q) @ direction)) < 1e-9)
[ok] [B-line] no other direction gives a smaller average loss [ok] [B-line] the one learned feature keeps almost all the variance the learned direction leaves an average loss of 3.65 squared degrees and keeps 98.4% of the total variance [ok] [B-line] the projection segment meets the line at a right angle [ok] [B-line] the projection segment meets the line at a right angle
Preview: the twenty days, an ellipse around each cluster, the line, and the orthogonal projection of two days on it.
fig, ax = plt.subplots(figsize=(6.2, 5.0))
ax.plot(X[:, 0], X[:, 1], "o", ms=5, mfc="none", color="black",
label="day (no label)")
for c, curve in enumerate(ell):
ax.plot(curve[:, 0], curve[:, 1], "-", lw=1.0, color="black",
label="cluster found by k-means" if c == 0 else None)
ax.plot(centroids[:, 0], centroids[:, 1], "s", ms=7, color="black",
label="learned centroid")
ax.plot(ends[:, 0], ends[:, 1], "--", lw=1.2, color="black",
label="learned direction")
seg = np.genfromtxt(OUT_DIR / "unsupervisedlearning_proj.csv",
delimiter=",", names=True)
ax.plot(seg["tmin"], seg["tmax"], ":", lw=1.0, color="black",
label="orthogonal projection")
ax.plot(feet[:, 0], feet[:, 1], "^", ms=6, color="black",
label="projected day")
ax.set_xlabel("daily minimum temperature (degrees C)")
ax.set_ylabel("daily maximum temperature (degrees C)")
ax.set_title("twenty unlabeled days at Krems: two centroids and one direction")
ax.legend(frameon=False, fontsize=8, loc="upper left")
fig.tight_layout()
fig.savefig(OUT_DIR / "unsupervisedlearning.png", dpi=110)
plt.close(fig)
check("[B-fig] the preview was written",
(OUT_DIR / "unsupervisedlearning.png").exists())
bad = [n for n, ok in report if not ok]
print(f"\n{len(report) - len(bad)}/{len(report)} checks pass")
if bad:
raise SystemExit("FAILED: " + "; ".join(bad))
[ok] [B-fig] the preview was written 12/12 checks pass
B-fig writes when the script runs