B-limit writes when the script runsDictionary of Applied Machine Learning · baseline
Numerical companion to the entry baseline: it recomputes what the entry states and prints one line per check
Numerical companion to the glossary entry 'baseline'. The GeoSphere Austria weather station Krems (station id 3805, 48.42 N, 15.62 E) records the minimum and maximum air temperature of each day; this script downloads the records for all 366 days of 2024 from the GeoSphere data hub (dataset klima-v2-1d) and writes them to baseline_weather.csv, so the exact numbers behind the figure stay on record. Checks pin the downloaded values to the 2024 archive, so a change on the server side is caught rather than silently absorbed.
Run it without installing anything:uv run https://dictionaryofml.org/terms/baseline.py
uv downloads this script and the pinned NumPy and Matplotlib it needs, then runs it; the script fetches any input file it uses. To keep the output files, download baseline.py into a folder and run uv run baseline.py there. With NumPy and Matplotlib already installed, python3 baseline.py, from any directory — it writes its output files into the current directory. Fixed seeds, so the printed numbers reproduce exactly. Download baseline.py · Notebook · Open in Colab
One cell per block of the script: the code, and what that code printed when it last ran here
"""A fundamental limit for the achievable loss, read off a weather record:
daily temperatures at Krems an der Donau, 2024.
Purpose
-------
Numerical companion to the glossary entry 'baseline'. The GeoSphere
Austria weather station Krems (station id 3805, 48.42 N, 15.62 E)
records the minimum and maximum air temperature of each day; this
script downloads the records for all 366 days of 2024 from the
GeoSphere data hub (dataset klima-v2-1d) and writes them to
baseline_weather.csv, so the exact numbers behind the figure stay on
record. Checks pin the downloaded values to the 2024 archive, so a
change on the server side is caught rather than silently absorbed.
Each day is one data point: its feature is the minimum temperature of
the day, its label the maximum temperature. The 15 days whose minimum
temperature lies between 11.5 and 12.5 degrees C carry maximum
temperatures from 13.2 to 33.4 degrees C: for (almost) the same
feature value, the label spreads over 20 degrees. Every hypothesis
maps (almost) the same feature value to (almost) the same prediction,
so this spread bounds the achievable average loss from below. The
strip-wise variance of the label, averaged over strips of the feature
axis that each hold at least four days, estimates that bound as 18.75
(squared degrees C); a hypothesis learned by linear regression reaches
an average squared error loss of 19.17 on the 366 days -- above the
estimated bound, as it must be, and within three percent of it, so the
learned hypothesis is already close to optimal. That comparison against a baseline is exactly what the entry
is about. A second block adds the baseline from the other side, the
one a method has to beat: predicting a single constant for every day,
which for a classification question means answering with the most
frequent label.
Deterministic: no randomness (the linear hypothesis is the closed-form
least-squares solution). Self-contained: numpy + matplotlib only
(stdlib urllib for the download).
Blocks
------
[B-fetch] Download the daily minimum and maximum temperature at Krems
for 2024 from the GeoSphere data hub; write the 366 records
to baseline_weather.csv and check them against the archive.
[B-spread] The 15 days with minimum temperature in [11.5, 12.5): their
maximum temperatures range from 13.2 to 33.4 degrees C, and
two of them share the exact feature value 11.8 with labels
13.2 and 29.9 -- no hypothesis can predict both correctly.
Write the band days and the remaining days to separate CSVs.
[B-limit] Estimate the smallest achievable average squared error loss
by the strip-wise variance of the label (18.75). The strips
start from a 1-degree grid and are merged until each holds at
least four days, so they vary in width and together cover
every day. Learn a linear hypothesis and check that its
average squared error loss (19.17) sits above that estimate
and within three percent of it. Writes the strips for the
entry's figure.
[B-trivial] The baseline from the other side: predicting one constant for
every day. For the squared error loss the best constant is
the mean label (17.91 degrees C, average loss 84.64); for the
classification question whether the maximum reaches 25
degrees C, answering no every day is right on 267 of the 366
days (accuracy 0.730). The level rises with the rarity of
the label: 0.888 for the 30-degree question.
Outputs
-------
baseline_weather.csv : date, tmin, tmax of the 366 downloaded days
baseline_strips.csv : lo, ymin, ymax of each strip of the feature axis
baseline_band.csv : tmin, tmax of the 15 days in the band [11.5, 12.5)
baseline_points.csv : tmin, tmax of the remaining 351 days
baseline_fit.csv : tmin, tmax along the learned linear hypothesis
baseline.png : preview (checking only) -- the 366 data points,
the band, its label spread, and the learned
linear hypothesis
"""
import json
import urllib.request
from pathlib import Path
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
OUT_DIR = Path(__file__).parent
report = [] # collects (check name, pass/fail) pairs
def check(name, ok): # records and prints one verification
report.append((name, bool(ok)))
print(f" [{'ok' if ok else 'FAIL'}] {name}")
Download the daily minimum and maximum temperature at Krems for 2024 from the GeoSphere data hub; write the 366 records to baseline_weather.csv and check them against the archive.
URL = ("https://dataset.api.hub.geosphere.at/v1/station/historical/"
"klima-v2-1d?parameters=tlmin,tlmax&station_ids=3805"
"&start=2024-01-01&end=2024-12-31")
with urllib.request.urlopen(URL, timeout=120) as resp:
payload = json.load(resp)
params = payload["features"][0]["properties"]["parameters"]
records = [(stamp[:10], lo, hi)
for stamp, lo, hi in zip(payload["timestamps"],
params["tlmin"]["data"],
params["tlmax"]["data"])]
with open(OUT_DIR / "baseline_weather.csv", "w") as f:
f.write("date,tmin,tmax\n")
for day, lo, hi in records:
f.write(f"{day},{lo},{hi}\n")
check("[B-fetch] 366 days downloaded for 2024", len(records) == 366)
check("[B-fetch] no day is missing a temperature",
all(lo is not None and hi is not None for _, lo, hi in records))
check("[B-fetch] the record matches the archive (Jan 1: 2.6 to 9.6)",
records[0] == ("2024-01-01", 2.6, 9.6))
[ok] [B-fetch] 366 days downloaded for 2024 [ok] [B-fetch] no day is missing a temperature [ok] [B-fetch] the record matches the archive (Jan 1: 2.6 to 9.6)
The 15 days with minimum temperature in [11.5, 12.5): their maximum temperatures range from 13.2 to 33.4 degrees C, and two of them share the exact feature value 11.8 with labels 13.2 and 29.9 -- no hypothesis can predict both correctly. Write the band days and the remaining days to separate CSVs.
temps = np.array([[lo, hi] for _, lo, hi in records])
tmin, tmax = temps[:, 0], temps[:, 1]
band = (tmin >= 11.5) & (tmin < 12.5)
check("[B-spread] 15 days have their minimum temperature in [11.5, 12.5)",
int(band.sum()) == 15)
check("[B-spread] their maximum temperatures range from 13.2 to 33.4",
round(float(tmax[band].min()), 1) == 13.2
and round(float(tmax[band].max()), 1) == 33.4)
check("[B-spread] two days share the feature value 11.8, labels 13.2 / 29.9",
sorted(tmax[tmin == 11.8].tolist()) == [13.2, 29.9])
header = "tmin,tmax"
np.savetxt(OUT_DIR / "baseline_band.csv", temps[band], delimiter=",",
header=header, comments="", fmt="%.1f")
np.savetxt(OUT_DIR / "baseline_points.csv", temps[~band], delimiter=",",
header=header, comments="", fmt="%.1f")
[ok] [B-spread] 15 days have their minimum temperature in [11.5, 12.5) [ok] [B-spread] their maximum temperatures range from 13.2 to 33.4 [ok] [B-spread] two days share the feature value 11.8, labels 13.2 / 29.9
Estimate the smallest achievable average squared error loss by the strip-wise variance of the label (18.75). The strips start from a 1-degree grid and are merged until each holds at least four days, so they vary in width and together cover every day. Learn a linear hypothesis and check that its average squared error loss (19.17) sits above that estimate and within three percent of it. Writes the strips for the entry's figure.
# The strips start from a 1-degree grid and are merged with their right
# neighbour until each holds at least MIN_DAYS days, so every day enters
# exactly one strip and the sparse tails widen instead of being dropped.
MIN_DAYS = 4
grid = np.arange(-10.5, 25.5, 1.0)
strips, lo_edge, so_far = [], grid[0], 0
for k in range(len(grid) - 1):
so_far += int(((tmin >= grid[k]) & (tmin < grid[k + 1])).sum())
if so_far >= MIN_DAYS:
strips.append((lo_edge, grid[k + 1]))
lo_edge, so_far = grid[k + 1], 0
if so_far: # absorb a short last strip
strips[-1] = (strips[-1][0], grid[-1])
floor_sum, floor_days = 0.0, 0 # bound estimate: strip-wise variance
strip_rows = []
for lo, hi in strips:
in_strip = (tmin >= lo) & (tmin < hi)
n = int(in_strip.sum())
floor_sum += float(tmax[in_strip].var(ddof=1)) * n
floor_days += n
strip_rows.append((lo, float(tmax[in_strip].min()),
float(tmax[in_strip].max())))
floor = floor_sum / floor_days
strip_rows.append((strips[-1][1], strip_rows[-1][1], strip_rows[-1][2]))
check(f"[B-limit] every strip holds at least {MIN_DAYS} days",
all(int(((tmin >= lo) & (tmin < hi)).sum()) >= MIN_DAYS
for lo, hi in strips))
check("[B-limit] the strips partition the record: every day is in one",
floor_days == len(tmin))
check("[B-limit] the strips vary in width",
len({round(hi - lo, 1) for lo, hi in strips}) > 1)
with open(OUT_DIR / "baseline_strips.csv", "w") as fh:
fh.write("lo,ymin,ymax\n")
for lo, ylo, yhi in strip_rows:
fh.write(f"{lo:.1f},{ylo:.1f},{yhi:.1f}\n")
A = np.stack([tmin, np.ones(len(tmin))], 1)
w, *_ = np.linalg.lstsq(A, tmax, rcond=None)
mse = float(np.mean((tmax - A @ w) ** 2))
print(f" learned hypothesis: tmax = {w[0]:.3f} * tmin + {w[1]:.3f}")
print(f" average squared error loss {mse:.2f}, "
f"estimated bound {floor:.2f} (over {len(strips)} strips, "
f"{floor_days} days)")
check("[B-limit] the bound estimate is 18.75 squared degrees C",
round(floor, 2) == 18.75)
check("[B-limit] the learned hypothesis reaches average loss 19.17",
round(mse, 2) == 19.17)
check("[B-limit] the learned hypothesis does not beat the bound, and comes "
"within three percent of it", floor <= mse < 1.03 * floor)
t_line = np.array([tmin.min(), tmin.max()])
np.savetxt(OUT_DIR / "baseline_fit.csv",
np.stack([t_line, w[0] * t_line + w[1]], 1), delimiter=",",
header=header, comments="", fmt="%.3f")
# ---- outputs: preview PNG (checking only)
fig, ax = plt.subplots(figsize=(6.4, 4.2))
for lo, hi in strips: # the strips behind everything
in_strip = (tmin >= lo) & (tmin < hi)
ax.add_patch(plt.Rectangle(
(lo, float(tmax[in_strip].min())), hi - lo,
float(tmax[in_strip].max() - tmax[in_strip].min()),
facecolor="0.88", edgecolor="0.6", lw=0.4, zorder=0))
ax.plot([], [], "s", color="0.88", mec="0.6",
label=f"strips of >= {MIN_DAYS} days, label range")
ax.plot(tmin[~band], tmax[~band], "o", color="gray", ms=3, mew=0,
alpha=0.6, label="351 other days")
ax.plot(tmin[band], tmax[band], "^", mfc="none", mec="tab:red", ms=7,
mew=1.5, label="15 days with tmin in [11.5, 12.5)")
ax.plot(t_line, w[0] * t_line + w[1], "k-", lw=1.5,
label="hypothesis learned by linear regression")
ax.annotate("", xy=(12.0, 33.4), xytext=(12.0, 13.2),
arrowprops=dict(arrowstyle="<->", lw=1.2))
ax.text(13.5, 21.0, "spread of 20.2 °C\nat the same feature value",
fontsize=8)
ax.set_xlabel("minimum temperature of the day (feature, °C)")
ax.set_ylabel("maximum temperature of the day (label, °C)")
ax.set_title("366 days at Krems, 2024: the label spread limits the loss")
ax.legend(frameon=False, fontsize=8, loc="upper left")
fig.tight_layout()
fig.savefig(OUT_DIR / "baseline.png", dpi=150)
[ok] [B-limit] every strip holds at least 4 days [ok] [B-limit] the strips partition the record: every day is in one [ok] [B-limit] the strips vary in width learned hypothesis: tmax = 1.099 * tmin + 10.022 average squared error loss 19.17, estimated bound 18.75 (over 28 strips, 366 days) [ok] [B-limit] the bound estimate is 18.75 squared degrees C [ok] [B-limit] the learned hypothesis reaches average loss 19.17 [ok] [B-limit] the learned hypothesis does not beat the bound, and comes within three percent of it predicting the constant 17.91 degrees C on every day: average squared error loss 84.64, 4.5 times the estimated bound 18.75
B-limit writes when the script runsThe baseline from the other side: predicting one constant for every day. For the squared error loss the best constant is the mean label (17.91 degrees C, average loss 84.64); for the classification question whether the maximum reaches 25 degrees C, answering no every day is right on 267 of the 366 days (accuracy 0.730). The level rises with the rarity of the label: 0.888 for the 30-degree question.
constant = float(tmax.mean())
mse_constant = float(np.mean((tmax - constant) ** 2))
print(f" predicting the constant {constant:.2f} degrees C on every day: "
f"average squared error loss {mse_constant:.2f}, "
f"{mse_constant / floor:.1f} times the estimated bound {floor:.2f}")
check("[B-trivial] the best constant is the mean label, 17.91 degrees C",
round(constant, 2) == 17.91)
check("[B-trivial] its average squared error loss is the label variance, 84.64",
round(mse_constant, 2) == 84.64
and abs(mse_constant - float(tmax.var())) < 1e-9)
check("[B-trivial] the learned hypothesis is four times better than that",
mse < 0.25 * mse_constant)
for degrees in (25.0, 30.0):
warm = tmax >= degrees # the label of the question
share = float(warm.mean())
accuracy = max(share, 1.0 - share) # answering with the majority
print(f" is the maximum at least {degrees:.0f} degrees C? "
f"{int(warm.sum())} of {len(warm)} days say yes; answering "
f"{'yes' if share > 0.5 else 'no'} every day gives accuracy "
f"{accuracy:.3f}, average zero-one loss {1.0 - accuracy:.3f}")
if degrees == 25.0:
acc25, warm25 = accuracy, warm
else:
acc30 = accuracy
check("[B-trivial] answering no on every day gets the 25-degree question "
"right on 267 of the 366 days, an accuracy of 0.730",
int((~warm25).sum()) == 267 and round(acc25, 3) == 0.730)
check("[B-trivial] the constant answer is right more often the rarer the "
"label: 0.888 for the 30-degree question", round(acc30, 3) == 0.888)
# one split of the days on the feature, for contrast with the constant answer
splits = np.unique(tmin)
scores = [( (tmin >= t_split).astype(int) == warm25.astype(int)).mean()
for t_split in splits]
best = int(np.argmax(scores))
print(f" one split of the days at a morning minimum of {splits[best]:.1f} "
f"degrees C answers the 25-degree question with accuracy "
f"{scores[best]:.3f}")
check("[B-trivial] a single split on the feature reaches accuracy 0.907, "
"well above the constant answer",
round(float(scores[best]), 3) == 0.907 and round(splits[best], 1) == 14.8)
failed = [name for name, ok in report if not ok]
print(f"{len(report) - len(failed)}/{len(report)} checks passed"
+ (f", FAILED: {failed}" if failed else ""))
[ok] [B-trivial] the best constant is the mean label, 17.91 degrees C [ok] [B-trivial] its average squared error loss is the label variance, 84.64 [ok] [B-trivial] the learned hypothesis is four times better than that is the maximum at least 25 degrees C? 99 of 366 days say yes; answering no every day gives accuracy 0.730, average zero-one loss 0.270 is the maximum at least 30 degrees C? 41 of 366 days say yes; answering no every day gives accuracy 0.888, average zero-one loss 0.112 [ok] [B-trivial] answering no on every day gets the 25-degree question right on 267 of the 366 days, an accuracy of 0.730 [ok] [B-trivial] the constant answer is right more often the rarer the label: 0.888 for the 30-degree question one split of the days at a morning minimum of 14.8 degrees C answers the 25-degree question with accuracy 0.907 [ok] [B-trivial] a single split on the feature reaches accuracy 0.907, well above the constant answer 18/18 checks passed