P-mechfeat writes when the script runsDictionary of Applied Machine Learning · feature
Numerical companion to the entry feature: it recomputes what the entry states and prints one line per check
One block per paragraph of the entry (marked [P...]): each block verifies numerically what the corresponding statement asserts. Self-contained (numpy/matplotlib only), fixed seed.
Run it with python3 feature.py, from any directory — it writes its output files into the current directory. Requires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Download feature.py
One cell per block of the script: the code, and what that code printed when it last ran here
"""
feature.py — numerical companion to the glossary entry 'feature'.
One block per paragraph of the entry (marked [P...]): each block verifies
numerically what the corresponding statement asserts. Self-contained
(numpy/matplotlib only), fixed seed.
Blocks
------
[P-featmap] Features are assembled into a feature vector by a feature
map acting on the data point (its raw features): sampled
signal values of an audio-like waveform form the feature
vector x = Phi(z).
[P-transform] New features from arithmetic transformations of existing
ones: augmenting a scalar feature x with x^2 turns an
unlearnable quadratic relation into one a linear hypothesis map
fits (the average training loss collapses).
[P-dtft] DFT-magnitude features are unchanged by time shifts of the
signal, while raw signal-value features change — the
shift-invariance claim, checked numerically.
[P-activation] The activation of a neuron is a derived feature: a fixed
random one-layer network turns raw features into
activations, and a linear hypothesis map on these activations fits
a nonlinear relation better than on the raw features.
[P-mechfeat] The narrower mechanistic-interpretability sense: a feature
as the projection of the activation vector onto a
direction — the projection of activations onto a planted
direction recovers a planted concept (correlation with
the concept indicator is high).
Outputs
-------
feature.png : preview figure (checking only).
Data generated by pythondemos/feature.py.
"""
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
rng = np.random.default_rng(42)
report = []
def check(name, ok):
report.append((name, bool(ok)))
print(f" [{'ok' if ok else 'FAIL'}] {name}")
Features are assembled into a feature vector by a feature map acting on the data point (its raw features): sampled signal values of an audio-like waveform form the feature vector x = Phi(z).
print("[P-featmap] feature map: data point -> feature vector")
t = np.arange(64)
signal = np.sin(2 * np.pi * 5 * t / 64) + 0.3 * np.sin(2 * np.pi * 11 * t / 64)
phi = lambda z: z.copy() # signal values as features
x = phi(signal)
check("feature vector collects the d = 64 signal values", x.size == 64)
check("the map is deterministic: same data point, same features",
np.array_equal(phi(signal), x))
[P-featmap] feature map: data point -> feature vector [ok] feature vector collects the d = 64 signal values [ok] the map is deterministic: same data point, same features
New features from arithmetic transformations of existing ones: augmenting a scalar feature x with x^2 turns an unlearnable quadratic relation into one a linear hypothesis map fits (the average training loss collapses).
print("[P-transform] transformed features make new features")
m = 120
u = rng.uniform(-2, 2, m)
yq = 1.5 * u**2 + 0.1 * rng.normal(size=m) # quadratic relation
fit_err = lambda X: np.mean((yq - np.c_[X, np.ones(m)] @ np.linalg.lstsq(
np.c_[X, np.ones(m)], yq, rcond=None)[0]) ** 2)
err_raw = fit_err(u[:, None])
err_aug = fit_err(np.c_[u, u**2])
print(f" average training loss raw: {err_raw:.3f}, with x^2 feature: "
f"{err_aug:.4f}")
check("adding the x^2 feature collapses the average training loss",
err_aug < 0.05 * err_raw)
[P-transform] transformed features make new features
average training loss raw: 2.633, with x^2 feature: 0.0098
[ok] adding the x^2 feature collapses the average training loss
DFT-magnitude features are unchanged by time shifts of the signal, while raw signal-value features change — the shift-invariance claim, checked numerically.
print("[P-dtft] DFT magnitudes are shift-invariant features")
shifted = np.roll(signal, 17) # time shift
mag = lambda z: np.abs(np.fft.rfft(z))
check("raw signal-value features change under the shift",
not np.allclose(signal, shifted))
check("DFT-magnitude features are unchanged by the shift",
np.allclose(mag(signal), mag(shifted), atol=1e-10))
[P-dtft] DFT magnitudes are shift-invariant features [ok] raw signal-value features change under the shift [ok] DFT-magnitude features are unchanged by the shift
The activation of a neuron is a derived feature: a fixed random one-layer network turns raw features into activations, and a linear hypothesis map on these activations fits a nonlinear relation better than on the raw features.
print("[P-activation] neuron activations as derived features")
W1 = rng.normal(size=(40, 1))
b1 = rng.normal(size=40)
act = lambda X: np.maximum(W1 @ X.T + b1[:, None], 0).T # ReLU layer
err_act = np.mean((yq - np.c_[act(u[:, None]), np.ones(m)] @
np.linalg.lstsq(np.c_[act(u[:, None]), np.ones(m)],
yq, rcond=None)[0]) ** 2)
print(f" average training loss on activations: {err_act:.4f}")
check("linear hypothesis map on activations beats raw features",
err_act < 0.1 * err_raw)
[P-activation] neuron activations as derived features
average training loss on activations: 0.0081
[ok] linear hypothesis map on activations beats raw features
The narrower mechanistic-interpretability sense: a feature as the projection of the activation vector onto a direction — the projection of activations onto a planted direction recovers a planted concept (correlation with the concept indicator is high).
print("[P-mechfeat] feature = projection of activations onto a direction")
d_act = 30
concept = (rng.uniform(size=500) > 0.5).astype(float) # planted concept
direction = rng.normal(size=d_act); direction /= np.linalg.norm(direction)
acts = rng.normal(size=(500, d_act)) + 4.0 * concept[:, None] * direction
proj = acts @ direction # scalar feature
corr = np.corrcoef(proj, concept)[0, 1]
print(f" corr(projection, concept) = {corr:.2f}")
check("projection onto the direction recovers the planted concept",
corr > 0.8)
other = rng.normal(size=d_act); other -= (other @ direction) * direction
other /= np.linalg.norm(other)
check("projection onto an orthogonal direction does not",
abs(np.corrcoef(acts @ other, concept)[0, 1]) < 0.2)
# ------------------------------------------------------------ preview
fig, ax = plt.subplots(1, 2, figsize=(8.4, 3.0))
ax[0].plot(t, signal, "-", lw=1, label="signal")
ax[0].plot(t, shifted, ":", lw=1, label="shifted")
ax[0].legend(frameon=False); ax[0].set_title("[P-dtft] signals")
ax[1].plot(mag(signal), "o-", ms=3, label="|DFT| original")
ax[1].plot(mag(shifted), "x", ms=4, label="|DFT| shifted")
ax[1].legend(frameon=False); ax[1].set_title("equal magnitudes")
fig.tight_layout()
fig.savefig("feature.png", dpi=110)
print(f"\n{sum(ok for _, ok in report)}/{len(report)} checks passed")
assert all(ok for _, ok in report)
[P-mechfeat] feature = projection of activations onto a direction
corr(projection, concept) = 0.89
[ok] projection onto the direction recovers the planted concept
[ok] projection onto an orthogonal direction does not
8/8 checks passed
P-mechfeat writes when the script runs