B-cm writes when the script runsDictionary of Applied Machine Learning · confusion matrix
Numerical companion to the entry confusion matrix: it recomputes what the entry states and prints one line per check
Numerical companion to the glossary entry 'cm' (confusion matrix). The GeoSphere Austria weather station Krems (station id 3805, 48.42 N, 15.62 E) records the minimum and maximum air temperature of each day; this script downloads the records for February and April 2024 from the GeoSphere data hub (dataset klima-v2-1d) and writes them to cm_weather.csv, so the exact numbers behind every figure stay on record. Checks pin the downloaded values to the 2024 archive, so a change on the server side is caught rather than silently absorbed.
Run it with python3 cm.py, from any directory — it writes its output files into the current directory. Requires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Download cm.py · Notebook · Open in Colab
One cell per block of the script: the code, and what that code printed when it last ran here
"""Frost warning at Krems an der Donau: two classifiers with the same
accuracy, told apart only by their confusion matrices.
Purpose
-------
Numerical companion to the glossary entry 'cm' (confusion matrix). The
GeoSphere Austria weather station Krems (station id 3805, 48.42 N,
15.62 E) records the minimum and maximum air temperature of each day;
this script downloads the records for February and April 2024 from the
GeoSphere data hub (dataset klima-v2-1d) and writes them to
cm_weather.csv, so the exact numbers behind every figure stay on
record. Checks pin the downloaded values to the 2024 archive, so a
change on the server side is caught rather than silently absorbed.
Each of the 40 data points is one day: its features are the two
temperatures of that day, its label is whether the minimum temperature
of the FOLLOWING day stays above 0 degrees C. 34 of the 40 following
days stay above 0 degrees C and only 6 bring frost, so the constant
prediction "above 0" -- a baseline that ignores the features --
is correct on 34 of the 40 days: accuracy 0.85. A linear classifier
learned by logistic regression from the 40 data points reaches the same
accuracy 0.85. The two confusion matrices nevertheless differ: the
learned classifier detects one of the six frost days at the cost of one
false alarm, while the baseline detects none.
The two 20-day windows (February 9-28 and April 10-29, each day paired
with the following day) are chosen so that each month contributes three
frost days: February 2024 was exceptionally warm, and the April frosts
fall in the cold snap of April 19-26.
Deterministic: no randomness (the minimization starts from the zero
vector). Self-contained: numpy + matplotlib only (stdlib urllib for
the download).
Blocks
------
[B-fetch] Download the daily minimum and maximum temperature at Krems
for February and April 2024 from the GeoSphere data hub;
write the 59 records to cm_weather.csv and check them
against the archive.
[B-data] Build the 40 data points from the downloaded temperatures;
check the label counts: 34 next days above 0 degrees C,
6 with frost.
[B-learn] Learn a linear classifier from the 40 data points by logistic
regression; check that the minimization has converged.
[B-cm] Confusion matrices of the learned classifier and of the
always-above-0 baseline; check that both reach accuracy 0.85
and that only the confusion matrices distinguish them.
Outputs
-------
cm_weather.csv : date, tmin, tmax of the 59 downloaded days
cm_points_above.csv : tmin, tmax of the 34 days followed by a day above 0
cm_points_frost.csv : tmin, tmax of the 6 days followed by a frost day
cm_boundary.csv : tmin, tmax along the learned decision boundary
cm_counts.csv : the four entries of both confusion matrices
cm.png : preview (checking only) -- the 40 data points with
the decision boundary, and the two confusion
matrices
"""
import json
import urllib.request
from pathlib import Path
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
OUT_DIR = Path(__file__).parent
report = [] # collects (check name, pass/fail) pairs
def check(name, ok): # records and prints one verification
report.append((name, bool(ok)))
print(f" [{'ok' if ok else 'FAIL'}] {name}")
Download the daily minimum and maximum temperature at Krems for February and April 2024 from the GeoSphere data hub; write the 59 records to cm_weather.csv and check them against the archive.
URL = ("https://dataset.api.hub.geosphere.at/v1/station/historical/"
"klima-v2-1d?parameters=tlmin,tlmax&station_ids=3805"
"&start=2024-02-01&end=2024-04-30")
with urllib.request.urlopen(URL, timeout=120) as resp:
payload = json.load(resp)
params = payload["features"][0]["properties"]["parameters"]
records = [(stamp[:10], lo, hi)
for stamp, lo, hi in zip(payload["timestamps"],
params["tlmin"]["data"],
params["tlmax"]["data"])
if stamp[5:7] in ("02", "04")] # February and April only
with open(OUT_DIR / "cm_weather.csv", "w") as f:
f.write("date,tmin,tmax\n")
for day, lo, hi in records:
f.write(f"{day},{lo},{hi}\n")
check("[B-fetch] 59 days downloaded for February and April 2024",
len(records) == 59)
check("[B-fetch] the record matches the archive (Feb 1: -3.8 to 10.4)",
records[0] == ("2024-02-01", -3.8, 10.4))
[ok] [B-fetch] 59 days downloaded for February and April 2024 [ok] [B-fetch] the record matches the archive (Feb 1: -3.8 to 10.4)
Build the 40 data points from the downloaded temperatures; check the label counts: 34 next days above 0 degrees C, 6 with frost.
dates = [day for day, _, _ in records]
temps = np.array([[lo, hi] for _, lo, hi in records])
date_index = {d: i for i, d in enumerate(dates)}
predictor_days = ([date_index["2024-02-09"] + k for k in range(20)] +
[date_index["2024-04-10"] + k for k in range(20)])
idx = np.array(predictor_days)
X = temps[idx] # features: (tmin, tmax) of day t
y = np.where(temps[idx + 1, 0] > 0.0, 1.0, -1.0) # +1: next day above 0
above, frost = y > 0, y < 0
check("[B-data] 40 data points, 20 per month", len(y) == 40)
check("[B-data] 34 next days stay above 0 degrees C", int(above.sum()) == 34)
check("[B-data] 6 next days bring frost, 3 per month",
int(frost.sum()) == 6 and int(frost[:20].sum()) == 3)
[ok] [B-data] 40 data points, 20 per month [ok] [B-data] 34 next days stay above 0 degrees C [ok] [B-data] 6 next days bring frost, 3 per month
Learn a linear classifier from the 40 data points by logistic regression; check that the minimization has converged.
Xb = np.hstack([X, np.ones((len(y), 1))]) # append constant feature
w = np.zeros(3)
STEP, ITERS = 0.01, 200_000
for _ in range(ITERS):
grad = -(y[:, None] * Xb / (1 + np.exp(y * (Xb @ w)))[:, None]).mean(0)
w -= STEP * grad
check("[B-learn] the minimization has converged (tiny final update)",
float(np.linalg.norm(grad)) < 1e-4)
print(f" learned weights: {w[0]:+.3f} * tmin {w[1]:+.3f} * tmax {w[2]:+.3f}")
pred = np.where(Xb @ w > 0, 1.0, -1.0)
pred_baseline = np.ones(len(y)) # always "above 0"
def confusion(y_true, y_pred):
"""2x2 counts; rows: true above 0 / frost, columns: predicted."""
return np.array([[int(np.sum((y_true == a) & (y_pred == p)))
for p in (1.0, -1.0)] for a in (1.0, -1.0)])
cm_clf = confusion(y, pred)
cm_base = confusion(y, pred_baseline)
acc_clf = float(np.trace(cm_clf)) / len(y)
acc_base = float(np.trace(cm_base)) / len(y)
[ok] [B-learn] the minimization has converged (tiny final update) learned weights: +0.179 * tmin +0.360 * tmax -3.027 learned classifier: [[33, 1], [5, 1]] accuracy 0.85 baseline: [[34, 0], [6, 0]] accuracy 0.85
Confusion matrices of the learned classifier and of the always-above-0 baseline; check that both reach accuracy 0.85 and that only the confusion matrices distinguish them.
print(f" learned classifier: {cm_clf.tolist()} accuracy {acc_clf:.2f}")
print(f" baseline: {cm_base.tolist()} accuracy {acc_base:.2f}")
check("[B-cm] both reach accuracy 0.85",
abs(acc_clf - 0.85) < 1e-9 and abs(acc_base - 0.85) < 1e-9)
check("[B-cm] the learned classifier detects a frost day, one false alarm",
cm_clf[1, 1] == 1 and cm_clf[0, 1] == 1)
check("[B-cm] the baseline detects no frost day at all", cm_base[1, 1] == 0)
# ---- outputs: CSVs for the entry's pgfplots figure, preview PNG
header = "tmin,tmax"
np.savetxt(OUT_DIR / "cm_points_above.csv", X[above], delimiter=",",
header=header, comments="", fmt="%.1f")
np.savetxt(OUT_DIR / "cm_points_frost.csv", X[frost], delimiter=",",
header=header, comments="", fmt="%.1f")
t_line = np.linspace(-5.0, 14.0, 2)
boundary = np.stack([t_line, -(w[0] * t_line + w[2]) / w[1]], 1)
np.savetxt(OUT_DIR / "cm_boundary.csv", boundary, delimiter=",",
header=header, comments="", fmt="%.3f")
with open(OUT_DIR / "cm_counts.csv", "w") as f:
f.write("classifier,true,pred_above,pred_frost\n")
for name, cm in (("learned", cm_clf), ("baseline", cm_base)):
for row, true in zip(cm, ("above", "frost")):
f.write(f"{name},{true},{row[0]},{row[1]}\n")
fig, axes = plt.subplots(1, 3, figsize=(11, 3.4),
gridspec_kw={"width_ratios": [1.6, 1, 1]})
ax = axes[0]
ax.plot(X[above, 0], X[above, 1], "o", color="tab:blue", ms=5,
label="next day above 0 °C")
ax.plot(X[frost, 0], X[frost, 1], "^", mfc="none", mec="tab:red", ms=8,
mew=1.5, label="next day frost")
ax.plot(boundary[:, 0], boundary[:, 1], "k--", lw=1.2,
label="decision boundary")
ax.set_xlabel("minimum temperature of the day (°C)")
ax.set_ylabel("maximum temperature of the day (°C)")
ax.set_ylim(4, 31)
ax.set_title("40 days at Krems, Feb/Apr 2024")
ax.legend(frameon=False, fontsize=8)
for ax, name, cm, acc in ((axes[1], "learned classifier", cm_clf, acc_clf),
(axes[2], "always-above-0 baseline", cm_base,
acc_base)):
ax.imshow(cm, cmap="Greys", vmin=0, vmax=45)
for i in range(2):
for j in range(2):
ax.text(j, i, str(cm[i, j]), ha="center", va="center",
color="black" if cm[i, j] < 25 else "white")
ax.set_xticks([0, 1], ["above 0", "frost"])
ax.set_yticks([0, 1], ["above 0", "frost"])
ax.set_xlabel("predicted")
ax.set_ylabel("true")
ax.set_title(f"{name}\naccuracy {acc:.2f}")
fig.tight_layout()
fig.savefig(OUT_DIR / "cm.png", dpi=150)
failed = [name for name, ok in report if not ok]
print(f"{len(report) - len(failed)}/{len(report)} checks passed"
+ (f", FAILED: {failed}" if failed else ""))
[ok] [B-cm] both reach accuracy 0.85 [ok] [B-cm] the learned classifier detects a frost day, one false alarm [ok] [B-cm] the baseline detects no frost day at all 9/9 checks passed
B-cm writes when the script runs