P-mechfeat writes when the script runsDictionary of Applied Machine Learning · feature
Numerical companion to the entry feature: it recomputes what the entry states and prints one line per check
One block per paragraph of the entry (marked [P...]): each block verifies numerically what the corresponding statement asserts. Self-contained (numpy/matplotlib only), fixed seed.
Run it without installing anything:uv run https://dictionaryofml.org/terms/feature.py
uv downloads this script and the pinned NumPy and Matplotlib it needs, then runs it; the script fetches any input file it uses. To keep the output files, download feature.py into a folder and run uv run feature.py there. With NumPy and Matplotlib already installed, python3 feature.py, from any directory — it writes its output files into the current directory. Fixed seeds, so the printed numbers reproduce exactly. Download feature.py · Notebook · Open in Colab
One cell per block of the script: the code, and what that code printed when it last ran here
"""
feature.py — numerical companion to the glossary entry 'feature'.
One block per paragraph of the entry (marked [P...]): each block verifies
numerically what the corresponding statement asserts. Self-contained
(numpy/matplotlib only), fixed seed.
Blocks
------
[P-featmap] Features are assembled into a feature vector by a feature
map acting on the data point (its raw features): sampled
signal values of an audio-like waveform form the feature
vector x = Phi(z).
[P-transform] New features from arithmetic transformations of existing
ones: augmenting a scalar feature x with x^2 turns an
unlearnable quadratic relation into one a linear hypothesis map
fits (the average training loss collapses).
[P-dtft] DFT-magnitude features are unchanged by time shifts of the
signal, while raw signal-value features change — the
shift-invariance claim, checked numerically.
[P-activation] The activation of a neuron is a derived feature: a fixed
random one-layer network turns raw features into
activations, and a linear hypothesis map on these activations fits
a nonlinear relation better than on the raw features.
[P-mechfeat] The narrower mechanistic-interpretability sense: a feature
as the projection of the activation vector onto a
direction — the projection of activations onto a planted
direction recovers a planted concept (correlation with
the concept indicator is high).
Outputs
-------
feature.png : preview figure (checking only).
Data generated by pythondemos/feature.py.
"""
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from pathlib import Path
OUT_DIR = Path(__file__).parent
rng = np.random.default_rng(42)
report = []
def check(name, ok):
report.append((name, bool(ok)))
print(f" [{'ok' if ok else 'FAIL'}] {name}")
Features are assembled into a feature vector by a feature map acting on the data point (its raw features): sampled signal values of an audio-like waveform form the feature vector x = Phi(z).
print("[P-featmap] feature map: data point -> feature vector")
t = np.arange(64)
signal = np.sin(2 * np.pi * 5 * t / 64) + 0.3 * np.sin(2 * np.pi * 11 * t / 64)
phi = lambda z: z.copy() # signal values as features
x = phi(signal)
check("feature vector collects the d = 64 signal values", x.size == 64)
check("the map is deterministic: same data point, same features",
np.array_equal(phi(signal), x))
[P-featmap] feature map: data point -> feature vector [ok] feature vector collects the d = 64 signal values [ok] the map is deterministic: same data point, same features
New features from arithmetic transformations of existing ones: augmenting a scalar feature x with x^2 turns an unlearnable quadratic relation into one a linear hypothesis map fits (the average training loss collapses).
print("[P-transform] transformed features make new features")
m = 120
u = rng.uniform(-2, 2, m)
yq = 1.5 * u**2 + 0.1 * rng.normal(size=m) # quadratic relation
fit_err = lambda X: np.mean((yq - np.c_[X, np.ones(m)] @ np.linalg.lstsq(
np.c_[X, np.ones(m)], yq, rcond=None)[0]) ** 2)
err_raw = fit_err(u[:, None])
err_aug = fit_err(np.c_[u, u**2])
print(f" average training loss raw: {err_raw:.3f}, with x^2 feature: "
f"{err_aug:.4f}")
check("adding the x^2 feature collapses the average training loss",
err_aug < 0.05 * err_raw)
[P-transform] transformed features make new features
average training loss raw: 2.633, with x^2 feature: 0.0098
[ok] adding the x^2 feature collapses the average training loss
DFT-magnitude features are unchanged by time shifts of the signal, while raw signal-value features change — the shift-invariance claim, checked numerically.
print("[P-dtft] DFT magnitudes are shift-invariant features")
shifted = np.roll(signal, 17) # time shift
mag = lambda z: np.abs(np.fft.rfft(z))
check("raw signal-value features change under the shift",
not np.allclose(signal, shifted))
check("DFT-magnitude features are unchanged by the shift",
np.allclose(mag(signal), mag(shifted), atol=1e-10))
[P-dtft] DFT magnitudes are shift-invariant features [ok] raw signal-value features change under the shift [ok] DFT-magnitude features are unchanged by the shift
The activation of a neuron is a derived feature: a fixed random one-layer network turns raw features into activations, and a linear hypothesis map on these activations fits a nonlinear relation better than on the raw features.
print("[P-activation] neuron activations as derived features")
W1 = rng.normal(size=(40, 1))
b1 = rng.normal(size=40)
act = lambda X: np.maximum(W1 @ X.T + b1[:, None], 0).T # ReLU layer
err_act = np.mean((yq - np.c_[act(u[:, None]), np.ones(m)] @
np.linalg.lstsq(np.c_[act(u[:, None]), np.ones(m)],
yq, rcond=None)[0]) ** 2)
print(f" average training loss on activations: {err_act:.4f}")
check("linear hypothesis map on activations beats raw features",
err_act < 0.1 * err_raw)
[P-activation] neuron activations as derived features
average training loss on activations: 0.0081
[ok] linear hypothesis map on activations beats raw features
The narrower mechanistic-interpretability sense: a feature as the projection of the activation vector onto a direction — the projection of activations onto a planted direction recovers a planted concept (correlation with the concept indicator is high).
print("[P-mechfeat] feature = projection of activations onto a direction")
d_act = 30
concept = (rng.uniform(size=500) > 0.5).astype(float) # planted concept
direction = rng.normal(size=d_act); direction /= np.linalg.norm(direction)
acts = rng.normal(size=(500, d_act)) + 4.0 * concept[:, None] * direction
proj = acts @ direction # scalar feature
corr = np.corrcoef(proj, concept)[0, 1]
print(f" corr(projection, concept) = {corr:.2f}")
check("projection onto the direction recovers the planted concept",
corr > 0.8)
other = rng.normal(size=d_act); other -= (other @ direction) * direction
other /= np.linalg.norm(other)
check("projection onto a perpendicular direction does not",
abs(np.corrcoef(acts @ other, concept)[0, 1]) < 0.2)
# ------------------------------------------------------------ preview
fig, ax = plt.subplots(1, 2, figsize=(8.4, 3.0))
ax[0].plot(t, signal, "-", lw=1, label="signal")
ax[0].plot(t, shifted, ":", lw=1, label="shifted")
ax[0].set_xlabel("sample index"); ax[0].set_ylabel("signal value")
ax[0].legend(frameon=False); ax[0].set_title("[P-dtft] signals")
ax[1].plot(mag(signal), "o-", ms=3, label="|DFT| original")
ax[1].plot(mag(shifted), "x", ms=4, label="|DFT| shifted")
ax[1].set_xlabel("frequency index"); ax[1].set_ylabel("DFT magnitude")
ax[1].legend(frameon=False); ax[1].set_title("equal magnitudes")
fig.tight_layout()
fig.savefig(OUT_DIR / "feature.png", dpi=110)
print(f"\n{sum(ok for _, ok in report)}/{len(report)} checks passed")
assert all(ok for _, ok in report)
[P-mechfeat] feature = projection of activations onto a direction
corr(projection, concept) = 0.89
[ok] projection onto the direction recovers the planted concept
[ok] projection onto a perpendicular direction does not
8/8 checks passed
P-mechfeat writes when the script runs