B-picture writes when the script runsDictionary of Applied Machine Learning · decision tree
Numerical companion to the entry decision tree: it recomputes what the entry states and prints one line per check
A single depth-2 regression tree is trained, in the CART fashion the entry describes, on actual weather measurements from the open data server of the Finnish Meteorological Institute (FMI): the 366 days of 2024 at the Helsinki Kaisaniemi station, with the daily minimum temperature as the feature and the daily maximum temperature as the label. Growing the tree greedily — each split chooses the threshold that most reduces the variance of the labels routed to the children — yields a piecewise constant hypothesis with four pieces, one per leaf node.
Run it with python3 decisiontree.py, from any directory — it writes its output files into the current directory. Requires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Download decisiontree.py · Notebook · Open in Colab
One cell per block of the script: the code, and what that code printed when it last ran here
"""decisiontree.py — numerical companion to the glossary entry
'decisiontree' (decision tree).
A single depth-2 regression tree is trained, in the CART fashion the
entry describes, on actual weather measurements from the open data
server of the Finnish Meteorological Institute (FMI): the 366 days of
2024 at the Helsinki Kaisaniemi station, with the daily minimum
temperature as the feature and the daily maximum temperature as the
label. Growing the tree greedily — each split chooses the threshold
that most reduces the variance of the labels routed to the children —
yields a piecewise constant hypothesis with four pieces, one per leaf
node.
Blocks
------
[B-fetch] Download the 366 daily temperature pairs for 2024 from the
FMI open data server and write them to
decisiontree_weather.csv; check the count and one pinned
measurement.
[B-tree] Grow a depth-2 regression tree by greedy variance
splitting; check that the hypothesis is piecewise constant
with at most four pieces, that each split reduced the
variance, and that the tree predicts the label far better
than a constant.
[B-picture] Write the scatterplot and the step-function curve read by
the entry's figure (decisiontree_scatter.csv,
decisiontree_curve.csv).
Deterministic: historical measurements, greedy splits, no randomness.
Self-contained: numpy + matplotlib only (stdlib urllib for the
download).
Outputs
-------
decisiontree_weather.csv : day, tmin, tmax for the 366 days of 2024
decisiontree_scatter.csv : tmin, tmax of the 366 days
decisiontree_curve.csv : tmin, pred -- the tree's step function
decisiontree.png : preview (checking only) -- the scatterplot
with the depth-2 tree's step function
"""
import re
import urllib.request
from pathlib import Path
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
OUT_DIR = Path(__file__).parent
report = []
def check(name, ok):
report.append((name, bool(ok)))
print(f" [{'ok' if ok else 'FAIL'}] {name}")
Download the 366 daily temperature pairs for 2024 from the FMI open data server and write them to decisiontree_weather.csv; check the count and one pinned measurement.
URL = ("https://opendata.fmi.fi/wfs?service=WFS&version=2.0.0"
"&request=getFeature"
"&storedquery_id=fmi::observations::weather::daily::simple"
"&fmisid=100971&starttime=2024-01-01T00:00:00Z"
"&endtime=2024-12-31T00:00:00Z¶meters=tmin,tmax")
with urllib.request.urlopen(URL, timeout=120) as resp:
xml = resp.read().decode()
triples = re.findall(
r"<BsWfs:Time>(\S+?)T.*?</BsWfs:Time>\s*"
r"<BsWfs:ParameterName>(\w+)</BsWfs:ParameterName>\s*"
r"<BsWfs:ParameterValue>(\S+)</BsWfs:ParameterValue>", xml, re.S)
rows = {}
for day, name, value in triples:
rows.setdefault(day, {})[name] = float(value)
days = sorted(rows)
tmin = np.array([rows[d]["tmin"] for d in days])
tmax = np.array([rows[d]["tmax"] for d in days])
with open(OUT_DIR / "decisiontree_weather.csv", "w") as f:
f.write("day,tmin,tmax\n")
for d in days:
f.write(f"{d},{rows[d]['tmin']:.1f},{rows[d]['tmax']:.1f}\n")
check("[B-fetch] 366 daily temperature pairs downloaded for 2024",
len(days) == 366)
check("[B-fetch] the record matches the archive "
"(May 2: 4.5 to 14.5 degrees)",
rows["2024-05-02"]["tmin"] == 4.5
and rows["2024-05-02"]["tmax"] == 14.5)
[ok] [B-fetch] 366 daily temperature pairs downloaded for 2024
[ok] [B-fetch] the record matches the archive (May 2: 4.5 to 14.5 degrees)
thresholds [np.float64(-10.35), np.float64(6.2), np.float64(10.55)], leaf values [np.float64(-9.0), np.float64(4.9), np.float64(14.7), np.float64(21.9)]
average squared error: tree 13.7, constant 96.1
Grow a depth-2 regression tree by greedy variance splitting; check that the hypothesis is piecewise constant with at most four pieces, that each split reduced the variance, and that the tree predicts the label far better than a constant.
MINLEAF = 20
def best_split(x, y):
"""Threshold minimizing the summed squared error of the two parts."""
order = np.argsort(x)
xs, ys = x[order], y[order]
best, best_sse = None, np.inf
for i in range(MINLEAF, len(xs) - MINLEAF + 1):
if xs[i - 1] == xs[i]:
continue
left, right = ys[:i], ys[i:]
sse = ((left - left.mean()) ** 2).sum() \
+ ((right - right.mean()) ** 2).sum()
if sse < best_sse:
best_sse, best = sse, (xs[i - 1] + xs[i]) / 2
return best, best_sse
root_t, root_sse = best_split(tmin, tmax)
cuts = [root_t]
for side in (tmin <= root_t, tmin > root_t):
t, _ = best_split(tmin[side], tmax[side])
if t is not None:
cuts.append(t)
cuts = sorted(cuts)
edges = [-np.inf] + cuts + [np.inf]
means = np.array([tmax[(tmin > lo) & (tmin <= hi)].mean()
for lo, hi in zip(edges[:-1], edges[1:])])
def predict(xq):
return means[np.searchsorted(np.array(cuts), xq)]
err_tree = float(np.mean((tmax - predict(tmin)) ** 2))
err_const = float(np.var(tmax))
print(f" thresholds {[round(c, 2) for c in cuts]}, "
f"leaf values {[round(m, 1) for m in means]}")
print(f" average squared error: tree {err_tree:.1f}, "
f"constant {err_const:.1f}")
check("[B-tree] the hypothesis is piecewise constant with at most "
"four pieces", len(means) <= 4)
check("[B-tree] the root split reduced the variance of the labels",
root_sse < err_const * len(tmax))
check("[B-tree] the tree predicts the label far better than a constant",
err_tree < err_const / 3)
[ok] [B-tree] the hypothesis is piecewise constant with at most four pieces [ok] [B-tree] the root split reduced the variance of the labels [ok] [B-tree] the tree predicts the label far better than a constant
Write the scatterplot and the step-function curve read by the entry's figure (decisiontree_scatter.csv, decisiontree_curve.csv).
np.savetxt(OUT_DIR / "decisiontree_scatter.csv",
np.stack([tmin, tmax], 1),
delimiter=",", header="tmin,tmax", comments="", fmt="%.1f")
bounds = [tmin.min() - 0.5] + cuts + [tmin.max() + 0.5]
gx, gy = [], []
for k, (lo, hi) in enumerate(zip(bounds[:-1], bounds[1:])):
gx += [lo, hi]
gy += [means[k], means[k]]
gx, gy = np.array(gx), np.array(gy)
np.savetxt(OUT_DIR / "decisiontree_curve.csv",
np.stack([gx, gy], 1),
delimiter=",", header="tmin,pred", comments="", fmt="%.2f")
check("[B-picture] the written curve has one constant level per leaf",
len(set(np.round(gy, 4))) == len(means))
fig, ax = plt.subplots(figsize=(6.4, 4.2))
ax.plot(tmin, tmax, "o", ms=2.5, color="0.6", label="days of 2024")
ax.plot(gx, gy, "-", lw=2.2, color="black",
label="depth-2 tree (step function)")
ax.set_xlabel("minimum temperature of the day (°C)")
ax.set_ylabel("maximum temperature of the day (°C)")
ax.set_title("a depth-2 regression tree at Kaisaniemi")
ax.legend(frameon=False, fontsize=8)
fig.tight_layout()
fig.savefig(OUT_DIR / "decisiontree.png", dpi=150)
failed = [name for name, ok in report if not ok]
print(f"{len(report) - len(failed)}/{len(report)} checks passed"
+ (f", FAILED: {failed}" if failed else ""))
[ok] [B-picture] the written curve has one constant level per leaf 6/6 checks passed
B-picture writes when the script runs