Dictionary of Applied Machine Learning · dataset

dataset — Python demo

Numerical companion to the entry dataset: it recomputes what the entry states and prints one line per check

One block per paragraph of the entry (marked [P...]): each block verifies numerically what the corresponding statement asserts. Self-contained (numpy/matplotlib only), fixed seed.

Run it without installing anything:
uv run https://dictionaryofml.org/terms/dataset.py
uv downloads this script and the pinned NumPy and Matplotlib it needs, then runs it; the script fetches any input file it uses. To keep the output files, download dataset.py into a folder and run uv run dataset.py there. With NumPy and Matplotlib already installed, python3 dataset.py, from any directory — it writes its output files into the current directory. Fixed seeds, so the printed numbers reproduce exactly. Download dataset.py · Notebook · Open in Colab

The script, block by block

One cell per block of the script: the code, and what that code printed when it last ran here

setup

"""
dataset.py — numerical companion to the glossary entry 'dataset'.

One block per paragraph of the entry (marked [P...]): each block verifies
numerically what the corresponding statement asserts. Self-contained
(numpy/matplotlib only), fixed seed.

Blocks
------
[P-setsample] Two readings of 'dataset': strictly a set of distinct data
              points (no order, no repetitions) vs the ML usage as a
              sample (an indexed sequence that may repeat). A sequence
              with a repeated data point has m = 4 entries but only 3
              distinct elements, and reordering the sequence leaves the
              underlying set unchanged.
[P-table]     The relational-model reading: a table whose rows are data
              points and whose columns are attributes (the cow table of
              the entry). ML methods use the attribute columns as
              features or the label; the row order is immaterial — the
              least-squares fit computed from a row-shuffled table is
              identical.

Outputs
-------
dataset.png : preview figure (checking only).

Data generated by pythondemos/dataset.py.
"""

import numpy as np
import matplotlib

matplotlib.use("Agg")
import matplotlib.pyplot as plt
from pathlib import Path

OUT_DIR = Path(__file__).parent

rng = np.random.default_rng(42)
report = []


def check(name, ok):
    report.append((name, bool(ok)))
    print(f"  [{'ok' if ok else 'FAIL'}] {name}")

P-setsample

Two readings of 'dataset': strictly a set of distinct data points (no order, no repetitions) vs the ML usage as a sample (an indexed sequence that may repeat). A sequence with a repeated data point has m = 4 entries but only 3 distinct elements, and reordering the sequence leaves the underlying set unchanged.

print("[P-setsample] set of distinct points vs indexed sample")
z1, z2, z3 = np.array([1.0, 2.0]), np.array([3.0, 1.0]), np.array([2.0, 2.0])
sample = np.stack([z1, z2, z1, z3])              # position 1 and 3 repeat
as_set = np.unique(sample, axis=0)
check("the sample has m = 4 indexed entries", sample.shape[0] == 4)
check("the underlying set has only 3 distinct data points",
      as_set.shape[0] == 3)
perm = rng.permutation(4)
check("reordering the sample leaves the set unchanged",
      np.array_equal(np.unique(sample[perm], axis=0), as_set))

[P-setsample] set of distinct points vs indexed sample
  [ok] the sample has m = 4 indexed entries
  [ok] the underlying set has only 3 distinct data points
  [ok] reordering the sample leaves the set unchanged

P-table

The relational-model reading: a table whose rows are data points and whose columns are attributes (the cow table of the entry). ML methods use the attribute columns as features or the label; the row order is immaterial — the least-squares fit computed from a row-shuffled table is identical.

print("[P-table] relational table: rows = data points, columns = attributes")
# the entry's cow table: Name, Weight, Age, Height, Stomach temperature
names = np.array(["Zenzi", "Berta", "Resi"])
table = np.array([[100.0, 4.0, 100.0, 25.0],
                  [140.0, 3.0, 130.0, 23.0],
                  [120.0, 4.0, 120.0, 31.0]])
X = table[:, [0, 1, 2]]                          # features: weight, age, height
y = table[:, 3]                                  # label: stomach temperature
check("each row is one data point, each column one attribute",
      table.shape == (3, 4))
w = np.linalg.lstsq(np.c_[X, np.ones(3)], y, rcond=None)[0]
perm = rng.permutation(3)
w_shuffled = np.linalg.lstsq(np.c_[X[perm], np.ones(3)], y[perm],
                             rcond=None)[0]
check("row order is immaterial: shuffled table gives the same fit",
      np.allclose(w, w_shuffled, atol=1e-8))
check("attribute domains bound the columns (weights within [100, 140])",
      X[:, 0].min() >= 100 and X[:, 0].max() <= 140)

# ------------------------------------------------------------ preview
fig, ax = plt.subplots(figsize=(5.2, 3.0))
ax.axis("off")
cell_text = [[n] + [f"{v:g}" for v in row]
             for n, row in zip(names, table)]
tab = ax.table(cellText=cell_text,
               colLabels=["Name", "Weight", "Age", "Height", "Temp."],
               loc="center")
tab.scale(1, 1.4)
ax.set_title("[P-table] dataset as a relation")
fig.tight_layout()
fig.savefig(OUT_DIR / "dataset.png", dpi=110)
print(f"\n{sum(ok for _, ok in report)}/{len(report)} checks passed")
assert all(ok for _, ok in report)

[P-table] relational table: rows = data points, columns = attributes
  [ok] each row is one data point, each column one attribute
  [ok] row order is immaterial: shuffled table gives the same fit
  [ok] attribute domains bound the columns (weights within [100, 140])

6/6 checks passed