Dictionary of Applied Machine Learning · covariance matrix

covariance matrix — Python demo

Updated on 2026-11-13

Numerical companion to the entry covariance matrix: it recomputes what the entry states and prints one line per check

The entry summarizes a dataset of daily weather measurements by the covariance matrix of the random vector obtained by drawing one day of the dataset uniformly at random. This demo builds that dataset from the GeoSphere Austria archive, computes the summary, draws the ellipse the entry's Fig. 1 shows, whitens the data for its Fig. 2, and checks what the entry claims about the precision matrix. Self-contained (numpy/matplotlib only, stdlib urllib for the download), fixed seed.

Run it without installing anything:
uv run https://dictionaryofml.org/terms/covmtx.py
uv downloads this script and the pinned NumPy and Matplotlib it needs, then runs it; the script fetches any input file it uses. To keep the output files, download covmtx.py into a folder and run uv run covmtx.py there. With NumPy and Matplotlib already installed, python3 covmtx.py, from any directory — it writes its output files into the current directory. Fixed seeds, so the printed numbers reproduce exactly. Download covmtx.py · Notebook · Open in Colab

Before running a downloaded copy, scan it: uvx bandit -q -t B102,B307,B602,B605 covmtx.py (B102,B307,B602,B605 are the exec, eval and shell checks of bandit) reports nothing: the script runs no shell command and evaluates no generated code.

The script, block by block

One cell per block of the script: the code, and what that code printed when it last ran here

setup

"""
covmtx.py — numerical companion to the glossary entry 'covariance matrix'.

The entry summarizes a dataset of daily weather measurements by the
covariance matrix of the random vector obtained by drawing one day of the
dataset uniformly at random. This demo builds that dataset from the
GeoSphere Austria archive, computes the summary, draws the ellipse the
entry's Fig. 1 shows, whitens the data for its Fig. 2, and checks what
the entry claims about the precision matrix. Self-contained
(numpy/matplotlib only, stdlib urllib for the download), fixed seed.

Blocks
------
[B-data]    The days of 2020 to 2024 at Krems an der Donau. A data point
            is one day, its feature vector the air temperature at 06:00
            and at 15:00 UTC, taken from the hourly archive. A day with
            either reading missing is dropped: 3 of the 1827 days, so
            1824 remain.
[B-cov]     Drawing a day uniformly at random gives a random vector whose
            mean vector is the sample mean and whose covariance matrix is
            the average outer product of the centered days: the variances
            of the two temperatures on the diagonal, their covariance off
            it. This is the sample covariance matrix in the 1/m
            convention; the unbiased 1/(m-1) convention differs by the
            factor m/(m-1), which is 1.0005 here.
[B-ellipse] The level set of the quadratic form in the inverse covariance
            matrix at the value 4 is an ellipse centered at the mean
            vector; its principal axes are the eigenvectors of the
            covariance matrix with half-lengths twice the square roots of
            the eigenvalues. Under a Gaussian model the ellipse would hold
            1 - exp(-2) = 86.5 percent of the days; the measured share is
            close to it.
[B-prec]    The precision matrix, the inverse of the covariance matrix:
            the reciprocal of its j-th diagonal entry is the conditional
            variance of entry j given the other entry, which for two
            entries is the variance of that entry times one minus the
            squared correlation. It is the smallest risk of predicting
            the afternoon temperature from the morning one under the
            squared error loss, attained by the best linear prediction.
[B-whiten]  Multiplying each centered feature vector by the inverse square
            root of the covariance matrix gives transformed feature
            vectors with zero mean and the identity as covariance matrix;
            the ellipse becomes the circle of radius two.

Outputs
-------
covmtx_days_plot.csv  : t06, t15 of 304 of the 1824 days (every 6th in
                        date order), the scatter of Fig. 1 and Fig. 2(a).
covmtx_ellipse.csv    : the ellipse of Fig. 1 as a closed polyline.
covmtx_mean.csv       : the mean vector, one row.
covmtx_white_plot.csv : w1, w2, the same 304 days after whitening, the
                        scatter of Fig. 2(b).
covmtx.png            : preview figure (checking only).

Data generated by pythondemos/covmtx.py.
"""

# /// script
# requires-python = ">=3.10"
# dependencies = ["matplotlib==3.10.7", "numpy==2.3.4"]
# ///

import json
import urllib.request

import numpy as np
import matplotlib

matplotlib.use("Agg")
import matplotlib.pyplot as plt
from pathlib import Path

OUT_DIR = Path(__file__).parent
ARCHIVE = "https://dataset.api.hub.geosphere.at/v1/station/historical/"
STATION = 3805                      # Krems an der Donau
HOURS = ("06:00", "15:00")          # the two readings of a day (UTC)
EVERY = 6                           # every 6th day is drawn: 304 of 1824
LEVEL = 4.0                         # the level set the entry draws
SEED = 20261029
report = []


def check(name, ok):
    report.append((name, bool(ok)))
    print(f"  [{'ok' if ok else 'FAIL'}] {name}")


def fetch_hourly(parameter, start, end):
    """One hourly parameter of one station from the GeoSphere archive."""
    url = (f"{ARCHIVE}klima-v2-1h?parameters={parameter}"
           f"&station_ids={STATION}&start={start}&end={end}")
    with urllib.request.urlopen(url, timeout=300) as resp:
        payload = json.load(resp)
    values = payload["features"][0]["properties"]["parameters"][parameter]
    return (np.array(values["data"], dtype=float),
            [t[:16] for t in payload["timestamps"]])


def write_rows(name, header, rows):
    with open(OUT_DIR / name, "w") as f:
        f.write(header + "\n")
        for r in rows:
            f.write(",".join(f"{v:.3f}" for v in r) + "\n")

B-data

The days of 2020 to 2024 at Krems an der Donau. A data point is one day, its feature vector the air temperature at 06:00 and at 15:00 UTC, taken from the hourly archive. A day with either reading missing is dropped: 3 of the 1827 days, so 1824 remain.

temp, stamps = fetch_hourly("tl", "2020-01-01T00:00", "2024-12-31T23:00")
by_hour = {h: {s[:10]: v for s, v in zip(stamps, temp) if s[11:] == h}
           for h in HOURS}
dates = sorted(by_hour[HOURS[0]])
pairs = np.array([[by_hour[h][d] for h in HOURS] for d in dates])
complete = ~np.isnan(pairs).any(axis=1)
dropped = [d for d, c in zip(dates, complete) if not c]
days = pairs[complete]
m = days.shape[0]
print(f"[B-data] {len(dates)} days at Krems an der Donau, 2020 to 2024, "
      f"read at {HOURS[0]} and {HOURS[1]} UTC; {len(dropped)} days with a "
      f"missing reading dropped ({', '.join(dropped)}); {m} days kept; "
      f"temperature at {HOURS[0]} from {days[:, 0].min():.1f} to "
      f"{days[:, 0].max():.1f}, at {HOURS[1]} from {days[:, 1].min():.1f} "
      f"to {days[:, 1].max():.1f} degrees")
check("[B-data]    five full years, two leap years, one reading per day "
      "and hour", len(dates) == 1827
      and all(len(by_hour[h]) == 1827 for h in HOURS))
check("[B-data]    1824 days carry both readings, none is missing after "
      "the drop", m == 1824 and not np.isnan(days).any())
[B-data] 1827 days at Krems an der Donau, 2020 to 2024, read at 06:00 and 15:00 UTC; 3 days with a missing reading dropped (2020-05-05, 2023-12-31, 2024-01-12); 1824 days kept; temperature at 06:00 from -12.5 to 26.2, at 15:00 from -5.0 to 35.2 degrees
  [ok] [B-data]    five full years, two leap years, one reading per day and hour
  [ok] [B-data]    1824 days carry both readings, none is missing after the drop

B-cov

Drawing a day uniformly at random gives a random vector whose mean vector is the sample mean and whose covariance matrix is the average outer product of the centered days: the variances of the two temperatures on the diagonal, their covariance off it. This is the sample covariance matrix in the 1/m convention; the unbiased 1/(m-1) convention differs by the factor m/(m-1), which is 1.0005 here.

mean_vec = days.mean(axis=0)
centered = days - mean_vec
C = centered.T @ centered / m
rho = C[0, 1] / np.sqrt(C[0, 0] * C[1, 1])
print(f"[B-cov] mean vector ({mean_vec[0]:.2f}, {mean_vec[1]:.2f}) degrees; "
      f"covariance matrix [[{C[0, 0]:.2f}, {C[0, 1]:.2f}], "
      f"[{C[1, 0]:.2f}, {C[1, 1]:.2f}]]; correlation {rho:.3f}")
check("[B-cov]     drawing a day uniformly at random has the sample mean as "
      "its mean vector", np.allclose(mean_vec, days.mean(axis=0)))
check("[B-cov]     and the average outer product of the centered days as its "
      "covariance matrix", np.allclose(C, np.cov(days.T, bias=True)))
check("[B-cov]     whose diagonal holds the two variances",
      abs(C[0, 0] - days[:, 0].var()) < 1e-9
      and abs(C[1, 1] - days[:, 1].var()) < 1e-9)
check("[B-cov]     and whose off-diagonal entries are the covariance, equal "
      "to each other", abs(C[0, 1] - C[1, 0]) < 1e-12)
check("[B-cov]     the morning temperature varies less than the afternoon "
      "one, and the two covary positively", C[0, 0] < C[1, 1] and C[0, 1] > 0)
C_unbiased = centered.T @ centered / (m - 1)
print(f"[B-cov] the 1/(m-1) convention is larger by the factor "
      f"{m / (m - 1):.4f}: diagonal {C_unbiased[0, 0]:.2f} against "
      f"{C[0, 0]:.2f}")
check("[B-cov]     the two conventions differ by exactly the factor m/(m-1)",
      np.allclose(C_unbiased, C * m / (m - 1)))
check("[B-cov]     the covariance matrix is positive semi-definite",
      np.all(np.linalg.eigvalsh(C) > 0))
[B-cov] mean vector (9.34, 15.32) degrees; covariance matrix [[63.01, 65.40], [65.40, 85.72]]; correlation 0.890
  [ok] [B-cov]     drawing a day uniformly at random has the sample mean as its mean vector
  [ok] [B-cov]     and the average outer product of the centered days as its covariance matrix
  [ok] [B-cov]     whose diagonal holds the two variances
  [ok] [B-cov]     and whose off-diagonal entries are the covariance, equal to each other
  [ok] [B-cov]     the morning temperature varies less than the afternoon one, and the two covary positively
[B-cov] the 1/(m-1) convention is larger by the factor 1.0005: diagonal 63.05 against 63.01
  [ok] [B-cov]     the two conventions differ by exactly the factor m/(m-1)
  [ok] [B-cov]     the covariance matrix is positive semi-definite

B-ellipse

The level set of the quadratic form in the inverse covariance matrix at the value 4 is an ellipse centered at the mean vector; its principal axes are the eigenvectors of the covariance matrix with half-lengths twice the square roots of the eigenvalues. Under a Gaussian model the ellipse would hold 1 - exp(-2) = 86.5 percent of the days; the measured share is close to it.

C_inv = np.linalg.inv(C)
eigval, eigvec = np.linalg.eigh(C)
order = np.argsort(eigval)[::-1]
eigval, eigvec = eigval[order], eigvec[:, order]
angle = np.linspace(0.0, 2.0 * np.pi, 361)
unit = np.stack([np.cos(angle), np.sin(angle)], axis=1)
ellipse = mean_vec + (unit * np.sqrt(LEVEL * eigval)) @ eigvec.T
quad = np.einsum("ij,jk,ik->i", ellipse - mean_vec, C_inv, ellipse - mean_vec)
check("[B-ellipse] every point of the drawn curve satisfies the quadratic "
      "form equal to 4", np.allclose(quad, LEVEL))
inside = np.einsum("ij,jk,ik->i", centered, C_inv, centered) <= LEVEL
share = float(inside.mean())
gauss = 1.0 - np.exp(-LEVEL / 2.0)
print(f"[B-ellipse] the ellipse holds {share:.3f} of the days, against the "
      f"{gauss:.3f} a Gaussian model predicts; half-axes "
      f"{np.sqrt(LEVEL * eigval[0]):.2f} and "
      f"{np.sqrt(LEVEL * eigval[1]):.2f} degrees")
check("[B-ellipse] the measured share of days inside is close to the "
      "Gaussian value", abs(share - gauss) < 0.05)
check("[B-ellipse] the long axis points along the eigenvector of the larger "
      "eigenvalue", eigval[0] > eigval[1]
      and abs(eigvec[0, 0] * eigvec[1, 0]) > 0)
plot_idx = np.arange(0, m, EVERY)
write_rows("covmtx_days_plot.csv", "t06,t15",
           [(days[i, 0], days[i, 1]) for i in plot_idx])
write_rows("covmtx_ellipse.csv", "t06,t15", ellipse)
write_rows("covmtx_mean.csv", "t06,t15", [mean_vec])
print(f"[B-ellipse] {plot_idx.size} of the {m} days written for the scatter")
check("[B-ellipse] the scatter holds every sixth day", plot_idx.size == 304)
  [ok] [B-ellipse] every point of the drawn curve satisfies the quadratic form equal to 4
[B-ellipse] the ellipse holds 0.911 of the days, against the 0.865 a Gaussian model predicts; half-axes 23.73 and 5.65 degrees
  [ok] [B-ellipse] the measured share of days inside is close to the Gaussian value
  [ok] [B-ellipse] the long axis points along the eigenvector of the larger eigenvalue
[B-ellipse] 304 of the 1824 days written for the scatter
  [ok] [B-ellipse] the scatter holds every sixth day

B-prec

The precision matrix, the inverse of the covariance matrix: the reciprocal of its j-th diagonal entry is the conditional variance of entry j given the other entry, which for two entries is the variance of that entry times one minus the squared correlation. It is the smallest risk of predicting the afternoon temperature from the morning one under the squared error loss, attained by the best linear prediction.

cond_var = 1.0 / C_inv[1, 1]
print(f"[B-prec] the reciprocal of the second diagonal entry of the "
      f"precision matrix is {cond_var:.3f}, against the variance "
      f"{C[1, 1]:.3f} of the {HOURS[1]} temperature alone")
check("[B-prec]    the reciprocal of a diagonal entry of the precision "
      "matrix is the variance of that entry times one minus the squared "
      "correlation", abs(cond_var - C[1, 1] * (1 - rho ** 2)) < 1e-9)
slope = C[0, 1] / C[0, 0]
intercept = mean_vec[1] - slope * mean_vec[0]
risk = float(np.mean((days[:, 1] - (intercept + slope * days[:, 0])) ** 2))
print(f"[B-prec] predicting the {HOURS[1]} temperature from the {HOURS[0]} "
      f"one by the best linear rule, slope {slope:.3f} and intercept "
      f"{intercept:.3f}, leaves the risk {risk:.3f}")
check("[B-prec]    which is the smallest risk of predicting the afternoon "
      "temperature from the morning one under the squared error loss",
      abs(risk - cond_var) < 1e-9)
check("[B-prec]    and it is smaller than the risk of the best constant "
      "prediction, the variance", cond_var < C[1, 1])
[B-prec] the reciprocal of the second diagonal entry of the precision matrix is 17.846, against the variance 85.724 of the 15:00 temperature alone
  [ok] [B-prec]    the reciprocal of a diagonal entry of the precision matrix is the variance of that entry times one minus the squared correlation
[B-prec] predicting the 15:00 temperature from the 06:00 one by the best linear rule, slope 1.038 and intercept 5.622, leaves the risk 17.846
  [ok] [B-prec]    which is the smallest risk of predicting the afternoon temperature from the morning one under the squared error loss
  [ok] [B-prec]    and it is smaller than the risk of the best constant prediction, the variance

B-whiten

Multiplying each centered feature vector by the inverse square root of the covariance matrix gives transformed feature vectors with zero mean and the identity as covariance matrix; the ellipse becomes the circle of radius two.

C_inv_root = eigvec @ np.diag(eigval ** -0.5) @ eigvec.T
white = centered @ C_inv_root.T
C_white = white.T @ white / m
print(f"[B-whiten] inverse square root [[{C_inv_root[0, 0]:.4f}, "
      f"{C_inv_root[0, 1]:.4f}], [{C_inv_root[1, 0]:.4f}, "
      f"{C_inv_root[1, 1]:.4f}]]; whitened mean "
      f"({white[:, 0].mean():.1e}, {white[:, 1].mean():.1e}), whitened "
      f"covariance matrix [[{C_white[0, 0]:.3f}, {C_white[0, 1]:.3f}], "
      f"[{C_white[1, 0]:.3f}, {C_white[1, 1]:.3f}]]")
check("[B-whiten]  the inverse square root squares to the inverse",
      np.allclose(C_inv_root @ C_inv_root, C_inv))
check("[B-whiten]  the whitened days have zero mean and the identity as "
      "covariance matrix", np.allclose(white.mean(axis=0), 0.0)
      and np.allclose(C_white, np.eye(2)))
check("[B-whiten]  the ellipse becomes the circle of radius two",
      np.allclose(np.linalg.norm((ellipse - mean_vec) @ C_inv_root.T,
                                 axis=1), np.sqrt(LEVEL)))
write_rows("covmtx_white_plot.csv", "w1,w2",
           [(white[i, 0], white[i, 1]) for i in plot_idx])

# ------------------------------------------------------------- preview
fig, ax = plt.subplots(1, 3, figsize=(13.2, 3.9))
ax[0].plot(days[plot_idx, 0], days[plot_idx, 1], "o", mfc="none",
           color="0.45", ms=2.5, label=f"{plot_idx.size} of the {m} days")
ax[0].plot(ellipse[:, 0], ellipse[:, 1], "k-", label="the level set at 4")
ax[0].plot(mean_vec[0], mean_vec[1], "kx", ms=9, label="mean vector")
ax[0].set_xlabel(f"temperature at {HOURS[0]} UTC (degC)")
ax[0].set_ylabel(f"temperature at {HOURS[1]} UTC (degC)")
ax[0].set_title("[B-ellipse] the dataset and its summary", fontsize=10)
ax[0].legend(frameon=False, fontsize=8, loc="upper left")
grid = np.linspace(days[:, 0].min(), days[:, 0].max(), 50)
ax[1].plot(days[plot_idx, 0], days[plot_idx, 1], "o", mfc="none",
           color="0.45", ms=2.5, label="the same days")
ax[1].plot(grid, intercept + slope * grid, "k-",
           label="best linear prediction")
ax[1].axhline(mean_vec[1], color="0.3", linestyle="--",
              label="best constant prediction")
ax[1].set_xlabel(f"temperature at {HOURS[0]} UTC (degC)")
ax[1].set_ylabel(f"temperature at {HOURS[1]} UTC (degC)")
ax[1].set_title("[B-prec] predicting the afternoon from the morning",
                fontsize=10)
ax[1].legend(frameon=False, fontsize=8, loc="upper left")
ax[2].plot(white[plot_idx, 0], white[plot_idx, 1], "o", mfc="none",
           color="0.45", ms=2.5, label="the same days, whitened")
ax[2].plot(2.0 * np.cos(angle), 2.0 * np.sin(angle), "k-",
           label="the circle of radius two")
ax[2].plot(0.0, 0.0, "kx", ms=9, label="mean vector")
ax[2].set_xlabel("whitened feature 1")
ax[2].set_ylabel("whitened feature 2")
ax[2].set_aspect("equal")
ax[2].set_ylim(-2.8, 3.4)                 # room for the legend above the circle
ax[2].set_title("[B-whiten] after whitening", fontsize=10)
ax[2].legend(frameon=False, fontsize=8, loc="upper left")
fig.tight_layout()
fig.savefig(OUT_DIR / "covmtx.png", dpi=110)
print(f"\n{sum(ok for _, ok in report)}/{len(report)} checks passed")
assert all(ok for _, ok in report)
[B-whiten] inverse square root [[0.2421, -0.1328], [-0.1328, 0.1960]]; whitened mean (-1.1e-15, 2.0e-15), whitened covariance matrix [[1.000, -0.000], [-0.000, 1.000]]
  [ok] [B-whiten]  the inverse square root squares to the inverse
  [ok] [B-whiten]  the whitened days have zero mean and the identity as covariance matrix
  [ok] [B-whiten]  the ellipse becomes the circle of radius two

19/19 checks passed
Preview figure produced by covmtx.py
The preview figure the block B-whiten writes when the script runs