B-cdf writes when the script runsDictionary of Applied Machine Learning · inverse
Numerical companion to the entry inverse: it recomputes what the entry states and prints one line per check
Checks the entry's claims about inverses in the three settings it walks through: a function on a finite domain (the four shapes of its Fig. 1), the sigmoid and the logit of its opening paragraph, and the inverse of a linear map together with what replaces it when none exists. A last block draws a realization through the inverse of a cumulative distribution function. Self-contained (numpy/matplotlib only), fixed seed, no downloads.
Run it without installing anything:uv run https://dictionaryofml.org/terms/inverse.py
uv downloads this script and the pinned NumPy and Matplotlib it needs, then runs it; the script fetches any input file it uses. To keep the output files, download inverse.py into a folder and run uv run inverse.py there. With NumPy and Matplotlib already installed, python3 inverse.py, from any directory — it writes its output files into the current directory. Fixed seeds, so the printed numbers reproduce exactly. Download inverse.py · Notebook · Open in Colab
One cell per block of the script: the code, and what that code printed when it last ran here
"""
inverse.py — numerical companion to the glossary entry 'inverse'.
Purpose
-------
Checks the entry's claims about inverses in the three settings it walks
through: a function on a finite domain (the four shapes of its Fig. 1),
the sigmoid and the logit of its opening paragraph, and the inverse of a
linear map together with what replaces it when none exists. A last block
draws a realization through the inverse of a cumulative distribution
function. Self-contained (numpy/matplotlib only), fixed seed, no
downloads.
Blocks
------
[B-shapes] The two functions of Fig. 1 on the same four shapes. Sending
a shape to its number of corners is injective and lands on
every value of {0, 3, 4, 5}, so reading the table backwards
defines the inverse and recovers every shape. Recording only
whether a shape has a straight edge sends three of the four
shapes to "yes", so no table undoes it: the value "yes" has
three preimages.
[B-sigmoid] The logit undoes the sigmoid and the sigmoid undoes the
logit, both to within numerical tolerance over a grid, and the two
numbers the entry states hold: a score of 0 becomes the
probability 0.5, a score of 2.2 becomes 0.9.
[B-matrix] A square matrix of full rank: its inverse satisfies
A A^-1 = A^-1 A = I and its determinant is nonzero. A square
matrix whose columns are not linearly independent:
determinant zero, no inverse,
and the pseudoinverse satisfies A A^+ A = A while A^+ A is
not the identity. A matrix with more rows than columns, of full column
rank: a left inverse exists (A^+ A = I) and no right inverse does
(A A^+ is not the identity), which is the one-sided case the
entry describes.
[B-cdf] Inverse-CDF sampling: applying the inverse of the standard
Gaussian cdf to draws from the uniform distribution on [0,1]
returns a standard Gaussian sample, checked by its mean, its
spread, and the largest gap between its
empirical cdf and the target.
Outputs
-------
inverse.png : preview figure (checking only), two panels: the sigmoid
and the logit as reflections of each other in the line
y = x, and the empirical cdf of the inverse-cdf sample (thin
dashed) against the target cdf (thick gray).
Data generated by pythondemos/inverse.py.
"""
import math
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from pathlib import Path
OUT_DIR = Path(__file__).parent
SEED = 20261029
NR_DRAWS = 20000
report = []
def check(name, ok):
report.append((name, bool(ok)))
print(f" [{'ok' if ok else 'FAIL'}] {name}")
def sigmoid(z):
return 1.0 / (1.0 + np.exp(-np.asarray(z, dtype=float)))
def logit(p):
p = np.asarray(p, dtype=float)
return np.log(p / (1.0 - p))
def gaussian_cdf(z):
"""The standard Gaussian cdf, from the error function of the stdlib."""
return 0.5 * (1.0 + np.vectorize(math.erf)(np.asarray(z, dtype=float)
/ math.sqrt(2.0)))
# the grid that represents the cdf and, read the other way round, its inverse
Z_GRID = np.linspace(-8.0, 8.0, 4001)
CDF_GRID = gaussian_cdf(Z_GRID)
def gaussian_cdf_inverse(u):
"""The inverse of the Gaussian cdf, read off the same grid."""
return np.interp(np.asarray(u, dtype=float), CDF_GRID, Z_GRID)
The two functions of Fig. 1 on the same four shapes. Sending a shape to its number of corners is injective and lands on every value of {0, 3, 4, 5}, so reading the table backwards defines the inverse and recovers every shape. Recording only whether a shape has a straight edge sends three of the four shapes to "yes", so no table undoes it: the value "yes" has three preimages.
SHAPES = ["circle", "triangle", "square", "pentagon"]
corners = {"circle": 0, "triangle": 3, "square": 4, "pentagon": 5}
straight_edge = {"circle": "no", "triangle": "yes", "square": "yes",
"pentagon": "yes"}
values = [corners[u] for u in SHAPES]
print(f"[B-shapes] corners: {corners}")
check("[B-shapes] the number of corners is a different value for each of "
"the four shapes", len(set(values)) == len(SHAPES))
corners_inverse = {v: u for u, v in corners.items()}
check("[B-shapes] reading that table backwards is a function on "
"{0, 3, 4, 5} and recovers every shape",
len(corners_inverse) == len(SHAPES)
and all(corners_inverse[corners[u]] == u for u in SHAPES))
check("[B-shapes] and it is undone in the other direction too",
all(corners[corners_inverse[v]] == v for v in corners_inverse))
yes_preimage = [u for u in SHAPES if straight_edge[u] == "yes"]
print(f"[B-shapes] preimage of \"yes\": {yes_preimage}")
check("[B-shapes] the straight-edge function sends three of the four "
"shapes to the same value, so it is not injective",
len(yes_preimage) == 3)
check("[B-shapes] no function undoes it: whatever shape is assigned to "
"\"yes\", the other two are not recovered",
all(sum(1 for u in SHAPES if straight_edge[u] == straight_edge[g]) > 1
for g in yes_preimage))
[B-shapes] corners: {'circle': 0, 'triangle': 3, 'square': 4, 'pentagon': 5}
[ok] [B-shapes] the number of corners is a different value for each of the four shapes
[ok] [B-shapes] reading that table backwards is a function on {0, 3, 4, 5} and recovers every shape
[ok] [B-shapes] and it is undone in the other direction too
[B-shapes] preimage of "yes": ['triangle', 'square', 'pentagon']
[ok] [B-shapes] the straight-edge function sends three of the four shapes to the same value, so it is not injective
[ok] [B-shapes] no function undoes it: whatever shape is assigned to "yes", the other two are not recovered
The logit undoes the sigmoid and the sigmoid undoes the logit, both to within numerical tolerance over a grid, and the two numbers the entry states hold: a score of 0 becomes the probability 0.5, a score of 2.2 becomes 0.9.
z = np.linspace(-12.0, 12.0, 2401)
check("[B-sigmoid] the logit undoes the sigmoid on every score of the grid",
np.allclose(logit(sigmoid(z)), z, atol=1e-9))
p = np.linspace(1e-6, 1 - 1e-6, 2401)
check("[B-sigmoid] and the sigmoid undoes the logit on every probability",
np.allclose(sigmoid(logit(p)), p, atol=1e-12))
print(f"[B-sigmoid] a score of 0 becomes the probability "
f"{float(sigmoid(0.0)):.3f}, a score of 2.2 becomes "
f"{float(sigmoid(2.2)):.3f}")
check("[B-sigmoid] the two numbers of the opening paragraph hold",
abs(float(sigmoid(0.0)) - 0.5) < 1e-12
and abs(float(sigmoid(2.2)) - 0.9) < 0.005)
[ok] [B-sigmoid] the logit undoes the sigmoid on every score of the grid [ok] [B-sigmoid] and the sigmoid undoes the logit on every probability [B-sigmoid] a score of 0 becomes the probability 0.500, a score of 2.2 becomes 0.900 [ok] [B-sigmoid] the two numbers of the opening paragraph hold
A square matrix of full rank: its inverse satisfies A A^-1 = A^-1 A = I and its determinant is nonzero. A square matrix whose columns are not linearly independent: determinant zero, no inverse, and the pseudoinverse satisfies A A^+ A = A while A^+ A is not the identity. A matrix with more rows than columns, of full column rank: a left inverse exists (A^+ A = I) and no right inverse does (A A^+ is not the identity), which is the one-sided case the entry describes.
rng = np.random.default_rng(SEED)
A = rng.standard_normal((3, 3))
A_inv = np.linalg.inv(A)
eye = np.eye(3)
print(f"[B-matrix] determinant of the square matrix: {np.linalg.det(A):.4f}")
check("[B-matrix] a square matrix of full rank has an inverse with "
"A A^-1 = A^-1 A = I",
np.allclose(A @ A_inv, eye) and np.allclose(A_inv @ A, eye))
check("[B-matrix] and its determinant is nonzero",
abs(np.linalg.det(A)) > 1e-9 and np.linalg.matrix_rank(A) == 3)
singular = A.copy()
singular[:, 2] = singular[:, 0] + singular[:, 1] # a column that repeats
print(f"[B-matrix] determinant of the matrix with a dependent column: "
f"{np.linalg.det(singular):.2e}, rank "
f"{np.linalg.matrix_rank(singular)}")
check("[B-matrix] a square matrix whose columns are not linearly "
"independent has determinant zero and no inverse",
abs(np.linalg.det(singular)) < 1e-9
and np.linalg.matrix_rank(singular) == 2)
singular_pinv = np.linalg.pinv(singular)
check("[B-matrix] its pseudoinverse satisfies A A^+ A = A",
np.allclose(singular @ singular_pinv @ singular, singular))
check("[B-matrix] while A^+ A is not the identity, so it is no inverse",
not np.allclose(singular_pinv @ singular, eye))
tall = rng.standard_normal((5, 2))
tall_pinv = np.linalg.pinv(tall)
check("[B-matrix] a matrix with more rows than columns, of full column "
"rank, has a left inverse A^+ A = I",
np.allclose(tall_pinv @ tall, np.eye(2)))
check("[B-matrix] and no right inverse, since A A^+ is not the identity",
not np.allclose(tall @ tall_pinv, np.eye(5)))
[B-matrix] determinant of the square matrix: 0.8999 [ok] [B-matrix] a square matrix of full rank has an inverse with A A^-1 = A^-1 A = I [ok] [B-matrix] and its determinant is nonzero [B-matrix] determinant of the matrix with a dependent column: -5.70e-17, rank 2 [ok] [B-matrix] a square matrix whose columns are not linearly independent has determinant zero and no inverse [ok] [B-matrix] its pseudoinverse satisfies A A^+ A = A [ok] [B-matrix] while A^+ A is not the identity, so it is no inverse [ok] [B-matrix] a matrix with more rows than columns, of full column rank, has a left inverse A^+ A = I [ok] [B-matrix] and no right inverse, since A A^+ is not the identity
Inverse-CDF sampling: applying the inverse of the standard Gaussian cdf to draws from the uniform distribution on [0,1] returns a standard Gaussian sample, checked by its mean, its spread, and the largest gap between its empirical cdf and the target.
u = rng.uniform(0.0, 1.0, NR_DRAWS)
sample = gaussian_cdf_inverse(u)
ks = float(np.max(np.abs(np.sort(gaussian_cdf(np.sort(sample)))
- (np.arange(NR_DRAWS) + 0.5) / NR_DRAWS)))
print(f"[B-cdf] {NR_DRAWS} uniform draws through the inverse cdf: mean "
f"{sample.mean():.3f}, spread {sample.std(ddof=1):.3f}, "
f"largest gap to the target cdf {ks:.4f}")
check("[B-cdf] the inverse of the cdf turns uniform draws into a "
"standard Gaussian sample", abs(sample.mean()) < 0.03
and abs(sample.std(ddof=1) - 1.0) < 0.03)
check("[B-cdf] whose empirical cdf stays close to the target", ks < 0.02)
check("[B-cdf] and the cdf undoes the inverse cdf on the grid",
np.allclose(gaussian_cdf_inverse(gaussian_cdf(np.linspace(-3, 3, 101))),
np.linspace(-3, 3, 101), atol=1e-3))
# ------------------------------------------------------------- preview
fig, ax = plt.subplots(1, 2, figsize=(9.4, 3.8))
zz = np.linspace(-6.0, 6.0, 400)
pp = np.linspace(0.0025, 0.9975, 400)
ax[0].plot(zz, sigmoid(zz), "k-", label="sigmoid")
ax[0].plot(pp, logit(pp), "k--", label="logit")
ax[0].plot([-6, 6], [-6, 6], color="0.6", linestyle=":", label="the line y = x")
ax[0].set_xlim(-6, 6)
ax[0].set_ylim(-6, 6)
ax[0].set_xlabel("argument")
ax[0].set_ylabel("value")
ax[0].set_title("[B-sigmoid] each is the other mirrored in y = x", fontsize=10)
ax[0].legend(frameon=False, fontsize=8, loc="upper left")
order = np.sort(sample)
# the two agree so closely that one hides the other: the target is drawn
# thick and light, the sample thin and dashed on top of it
ax[1].plot(Z_GRID, CDF_GRID, color="0.7", linewidth=3.0, label="target cdf")
ax[1].plot(order, (np.arange(NR_DRAWS) + 0.5) / NR_DRAWS, "k--", linewidth=1.0,
label="empirical cdf of the sample")
ax[1].set_xlim(-4, 4)
ax[1].set_xlabel("value")
ax[1].set_ylabel("cumulative probability")
ax[1].set_title("[B-cdf] uniform draws through the inverse cdf", fontsize=10)
ax[1].legend(frameon=False, fontsize=8, loc="upper left")
fig.tight_layout()
fig.savefig(OUT_DIR / "inverse.png", dpi=110)
print(f"\n{sum(ok for _, ok in report)}/{len(report)} checks passed")
assert all(ok for _, ok in report)
[B-cdf] 20000 uniform draws through the inverse cdf: mean 0.003, spread 0.994, largest gap to the target cdf 0.0048 [ok] [B-cdf] the inverse of the cdf turns uniform draws into a standard Gaussian sample [ok] [B-cdf] whose empirical cdf stays close to the target [ok] [B-cdf] and the cdf undoes the inverse cdf on the grid 18/18 checks passed
B-cdf writes when the script runs