{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3", "language": "python" }, "language_info": { "name": "python" }, "colab": { "name": "innerproduct.ipynb" } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": "# inner product \u2014 Python demo\n\nNumerical companion to the entry [inner product](https://dictionaryofml.org/terms/innerproduct.html) of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org/): it recomputes what the entry states and prints one line per check.\n\nVerifies the entry's axioms, geometry, and ML tie-ins for the standard Euclidean inner product = x^T x' on R^d, and for the implicit inner products encoded by a Gaussian kernel. Self-contained (numpy/ matplotlib only; the optional network check inside [P-cities] uses the stdlib urllib to query OpenStreetMap and is skipped without network), fixed seed.\n\nRequires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Generated from [`pythondemos/innerproduct.py`](https://dictionaryofml.org/terms/innerproduct.py); CC BY 4.0." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "# Notebook shim: the script resolves output paths relative to __file__,\n# which a notebook kernel does not define; everything lands in the\n# working directory instead.\nimport os\n__file__ = os.path.join(os.getcwd(), \"innerproduct.py\")\nos.makedirs(\"pythondemos\", exist_ok=True)" }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "\"\"\"\ninnerproduct.py \u2014 numerical companion to the glossary entry 'inner product'.\n\nPurpose\n-------\nVerifies the entry's axioms, geometry, and ML tie-ins for the standard\nEuclidean inner product = x^T x' on R^d, and for the implicit\ninner products encoded by a Gaussian kernel. Self-contained (numpy/\nmatplotlib only; the optional network check inside [P-cities] uses the\nstdlib urllib to query OpenStreetMap and is skipped without network),\nfixed seed.\n\nBlocks\n------\nOne block per paragraph of the entry (marked [P...]), in the order the\nparagraphs appear: each block verifies numerically what its paragraph asserts.\n\n[P-array] The opening framing: a colour image and a sampled sensor signal\n both flatten into the numeric array that a data point carries as\n its feature vector.\n[P-axioms] The three defining properties hold for 1000 random triples and\n scalars (d = 5); the complex case satisfies conjugate symmetry,\n linearity in the first argument and positive-definiteness; and the\n E{x y} inner product on zero-mean RVs is another instance,\n where the cosine of the angle is the correlation coefficient.\n[P-norm] The induced norm sqrt() equals the Euclidean length and\n satisfies the triangle inequality; the induced metric satisfies\n the metric axioms.\n[P-cos] = ||x|| ||x'|| cos(theta) in R^2, and the Cauchy-Schwarz\n bound holds for 1000 random pairs.\n[P-project] The closest point of a subspace: the error is orthogonal to every\n vector of the subspace, and the Pythagorean identity holds.\n[P-convex] Projection onto a closed convex set satisfies the variational\n inequality <= 0 and is the nearest point.\n[P-linreg] Linear regression: the error is orthogonal to every column of the\n feature matrix and the predictions are the orthogonal projection\n of the label vector onto its column space; an orthogonal Q leaves\n inner products, ERM values and predictions unchanged.\n[P-weight] The weighted inner product x^T A x' satisfies the axioms and\n CHANGES the similarity ranking: for A = diag(4, 1/4) the candidate\n preferred under the standard inner product loses under it.\n[P-basis] Coordinates with respect to an orthonormal basis are inner\n products; declaring an arbitrary basis orthonormal defines an\n inner product, which equals the weighted one with A = (B B^T)^-1.\n[P-cities] The geometry of the entry's city figure: five European cities as\n vectors in R^2, drawn in an orthonormal frame that puts Helsinki\n on the horizontal line, with the dashed rulers hitting it at the\n projected lengths; a neuron's weighted input is maximized by the\n input aligned with its weights; and the pinned coordinates are\n re-checked against OpenStreetMap when the network is available.\n[P-kernel] The Gram matrix of a Gaussian kernel on 40 points is positive\n semi-definite: kernel evaluations are inner products between\n transformed feature vectors.\n[P-dist] The squared Euclidean distance expands into three inner products,\n which is why distance-based methods touch the feature vectors only\n through inner products.\n\nOutputs\n-------\ninnerproduct.png : matplotlib preview \u2014 scatter of against\n ||x|| ||x'|| cos(theta) on the identity line\n (checking only; the entry's figure is schematic TikZ).\n\"\"\"\n\nimport numpy as np\nimport matplotlib\n\nmatplotlib.use(\"Agg\")\nimport matplotlib.pyplot as plt\n\nfrom pathlib import Path\n\nOUT_DIR = Path(__file__).parent\n\nreport = []\n\n\ndef check(name, ok):\n report.append((name, bool(ok)))\n print(f\" [{'ok' if ok else 'FAIL'}] {name}\")\n\n\nrng = np.random.default_rng(0)" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-array]** The opening framing: a colour image and a sampled sensor signal both flatten into the numeric array that a data point carries as its feature vector." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "# a colour image and a sampled sensor signal both flatten to feature vectors\nimg = rng.random((4, 5, 3)) # 4 x 5 pixels, 3 channels\nx_img = img.reshape(-1) # the numeric array of features\ntt = np.linspace(0.0, 1.0, 64, endpoint=False)\nx_sig = np.sin(2 * np.pi * 5 * tt) # 64 amplitudes of a signal\nok_arr = (x_img.shape == (4 * 5 * 3,)\n and np.isclose(x_img[0], img[0, 0, 0])\n and np.isclose(x_img[-1], img[-1, -1, -1])\n and x_sig.shape == (64,))\ncheck(\"[P-array] image pixels and signal amplitudes flatten to a feature \"\n \"vector\", ok_arr)" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-axioms]** The three defining properties hold for 1000 random triples and scalars (d = 5); the complex case satisfies conjugate symmetry, linearity in the first argument and positive-definiteness; and the E{x y} inner product on zero-mean RVs is another instance, where the cosine of the angle is the correlation coefficient." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "ok_sym = ok_lin = ok_pd = True\nfor _ in range(1000):\n x, xp, xpp = rng.standard_normal((3, 5))\n b = rng.standard_normal()\n ok_sym &= np.isclose(x @ xp, xp @ x)\n ok_lin &= np.isclose((b * x + xpp) @ xp, b * (x @ xp) + xpp @ xp)\n ok_pd &= x @ x >= 0\nok_pd &= np.isclose(np.zeros(5) @ np.zeros(5), 0.0)\ncheck(\"[P-axioms] symmetry, linearity, positive-definiteness\",\n ok_sym and ok_lin and ok_pd)\n\n# the complex case = sum_j x_j conj(x'_j): conjugate symmetry,\n# linearity in the FIRST argument, positive-definiteness (as in the entry)\ndef cip(x, xp):\n return np.sum(x * np.conj(xp))\n\n\nok_csym = ok_clin = ok_cpd = True\nfor _ in range(1000):\n z, zp, zpp = (rng.standard_normal((3, 5))\n + 1j * rng.standard_normal((3, 5)))\n b = rng.standard_normal() + 1j * rng.standard_normal()\n ok_csym &= np.isclose(cip(z, zp), np.conj(cip(zp, z)))\n ok_clin &= np.isclose(cip(b * z + zpp, zp), b * cip(z, zp) + cip(zpp, zp))\n ok_cpd &= (np.isclose(cip(z, z).imag, 0.0) and cip(z, z).real >= 0)\nok_cpd &= np.isclose(cip(np.zeros(5), np.zeros(5)), 0.0)\ncheck(\"[P-axioms] conjugate symmetry, first-argument linearity, pd over C\",\n ok_csym and ok_clin and ok_cpd)\n\npw = np.array([0.10, 0.15, 0.20, 0.25, 0.18, 0.12]) # nonnegative weights, sum 1\n\n\ndef Ew(v): # weighted mean E{v}\n return pw @ v\n\n\nxr, yr = rng.standard_normal((2, 6)) * 1.5\nxr, yr = xr - Ew(xr), yr - Ew(yr) # zero-mean RVs\nipE = Ew(xr * yr) # = E{x y}\ncos_ang = ipE / np.sqrt(Ew(xr * xr) * Ew(yr * yr)) # cosine of the angle\n# correlation from its defining ratio (means subtracted explicitly)\ncov_xy = Ew(xr * yr) - Ew(xr) * Ew(yr)\nvar_x = Ew(xr * xr) - Ew(xr) ** 2\nvar_y = Ew(yr * yr) - Ew(yr) ** 2\ncorr = cov_xy / np.sqrt(var_x * var_y) # correlation coefficient\nok_corr = np.isclose(corr, cos_ang)\n# uncorrelated = orthogonal: subtract the projection to decorrelate\nyo = yr - (ipE / Ew(xr * xr)) * xr\ncheck(\"[P-axioms] correlation = cosine of the angle; uncorrelated = \"\n \"orthogonal\", ok_corr and np.isclose(Ew(xr * yo), 0.0))" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-norm]** The induced norm sqrt() equals the Euclidean length and satisfies the triangle inequality; the induced metric satisfies the metric axioms." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "ok_norm = True\nfor _ in range(1000):\n x, xp, xpp = rng.standard_normal((3, 5))\n ok_norm &= np.isclose(np.sqrt(x @ x), np.linalg.norm(x))\n ok_norm &= np.linalg.norm(x + xp) <= np.linalg.norm(x) \\\n + np.linalg.norm(xp) + 1e-12\n # metric axioms: symmetry and triangle inequality\n ok_norm &= np.isclose(np.linalg.norm(x - xp), np.linalg.norm(xp - x))\n ok_norm &= np.linalg.norm(x - xpp) <= np.linalg.norm(x - xp) \\\n + np.linalg.norm(xp - xpp) + 1e-12\ncheck(\"[P-norm] induced norm and metric\", ok_norm)" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-cos]** = ||x|| ||x'|| cos(theta) in R^2, and the Cauchy-Schwarz bound holds for 1000 random pairs." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "ok_cos = ok_cs = True\nips, cos_prods = [], []\nfor _ in range(1000):\n x, xp = rng.standard_normal((2, 2))\n theta = np.arctan2(xp[1], xp[0]) - np.arctan2(x[1], x[0])\n lhs = x @ xp\n rhs = np.linalg.norm(x) * np.linalg.norm(xp) * np.cos(theta)\n ok_cos &= np.isclose(lhs, rhs)\n ok_cs &= abs(lhs) <= np.linalg.norm(x) * np.linalg.norm(xp) + 1e-12\n ips.append(lhs), cos_prods.append(rhs)\ncheck(\"[P-cos] cos-theta identity and Cauchy-Schwarz bound\",\n ok_cos and ok_cs)" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-project]** The closest point of a subspace: the error is orthogonal to every vector of the subspace, and the Pythagorean identity holds." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "# the closest point of a subspace: orthogonality condition and Pythagoras\nUs = rng.standard_normal((6, 2)) # a 2-dim subspace of R^6\nvs = rng.standard_normal(6)\ncoef = np.linalg.lstsq(Us, vs, rcond=None)[0]\nvhat = Us @ coef # the closest point\nerr = vs - vhat\nok_orth = np.allclose(Us.T @ err, 0.0, atol=1e-10) # error orthogonal to U\nu_other = Us @ rng.standard_normal(2)\nok_pyth = np.isclose(np.linalg.norm(vs - u_other) ** 2,\n np.linalg.norm(err) ** 2\n + np.linalg.norm(vhat - u_other) ** 2)\ncheck(\"[P-project] closest point of a subspace: error is orthogonal to it, \"\n \"and Pythagoras holds\", ok_orth and ok_pyth)" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-convex]** Projection onto a closed convex set satisfies the variational inequality <= 0 and is the nearest point." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "# projection onto a closed convex set: the variational inequality\nvc = np.array([2.5, 1.0]) # a point outside the unit ball\nvhat_c = vc / np.linalg.norm(vc) # its projection onto the ball\npts = rng.standard_normal((4000, 2))\npts = pts[np.linalg.norm(pts, axis=1) <= 1.0] # points of the set C\nok_var = np.all((pts - vhat_c) @ (vc - vhat_c) <= 1e-12)\nok_near = np.all(np.linalg.norm(pts - vc, axis=1)\n >= np.linalg.norm(vhat_c - vc) - 1e-12)\ncheck(\"[P-convex] projection onto a closed convex set satisfies the \"\n \"variational inequality and is the nearest point\", ok_var and ok_near)" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-linreg]** Linear regression: the error is orthogonal to every column of the feature matrix and the predictions are the orthogonal projection of the label vector onto its column space; an orthogonal Q leaves inner products, ERM values and predictions unchanged." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "# orthogonal transformations preserve inner products, hence the linear-model\n# ERM objective values and predictions (the entry's invariance statement)\nm = 50\nX = rng.standard_normal((m, 3))\ny = X @ np.array([1.0, -2.0, 0.5]) + 0.1 * rng.standard_normal(m)\nQ, _ = np.linalg.qr(rng.standard_normal((3, 3))) # a random orthogonal Q\nw = rng.standard_normal(3)\nok_ip = np.allclose((X @ Q.T) @ (Q @ w), X @ w) # = \nf_orig = np.mean((y - X @ w) ** 2) # ERM objective at w\nf_rot = np.mean((y - (X @ Q.T) @ (Q @ w)) ** 2) # after rotating x and w\nw_hat = np.linalg.lstsq(X, y, rcond=None)[0] # ERM solution, original\nw_hat_rot = np.linalg.lstsq(X @ Q.T, y, rcond=None)[0] # ERM solution, rotated\nok_pred = np.allclose((X @ Q.T) @ w_hat_rot, X @ w_hat) # same predictions\ncheck(\"[P-linreg] orthogonal Q preserves inner products, ERM values, \"\n \"predictions\", ok_ip and np.isclose(f_orig, f_rot) and ok_pred)\n\n# the least-squares characterization: the error is orthogonal to every column\n# of the feature matrix, so the predictions are the orthogonal projection of\n# the label vector onto its column space\nresid = y - X @ w_hat\nok_normal = np.allclose(X.T @ resid, 0.0, atol=1e-9)\nPcol = X @ np.linalg.pinv(X) # projector onto col(X)\nok_proj = np.allclose(Pcol @ y, X @ w_hat)\ncheck(\"[P-linreg] linear regression: error orthogonal to every column, \"\n \"predictions are the projection of the label vector\",\n ok_normal and ok_proj)" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-weight]** The weighted inner product x^T A x' satisfies the axioms and CHANGES the similarity ranking: for A = diag(4, 1/4) the candidate preferred under the standard inner product loses under it." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "Aw = np.diag([4.0, 0.25]) # positive definite weights\n\n\ndef ipw(x, yv): # weighted inner product\n return x @ Aw @ yv\n\n\nxq = np.array([1.0, 1.0]) # query vector\nc1, c2 = np.array([0.9, 1.2]), np.array([1.3, 0.4]) # two candidates\nok_axioms = (np.isclose(ipw(c1, c2), ipw(c2, c1)) # symmetry\n and np.isclose(ipw(2.0 * c1 + c2, xq),\n 2.0 * ipw(c1, xq) + ipw(c2, xq)) # bilinearity\n and ipw(xq, xq) > 0) # positive definite\nok_swap = (xq @ c1 > xq @ c2) and (ipw(xq, c1) < ipw(xq, c2))\ncheck(\"[P-weight] weighted inner product: axioms hold, similarity \"\n \"ranking swaps\", ok_axioms and ok_swap)" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-basis]** Coordinates with respect to an orthonormal basis are inner products; declaring an arbitrary basis orthonormal defines an inner product, which equals the weighted one with A = (B B^T)^-1." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "# (i) coordinates w.r.t. an orthonormal basis = inner products with it\nQb, _ = np.linalg.qr(rng.standard_normal((4, 4))) # orthonormal basis (cols)\nu4 = rng.standard_normal(4)\ncoords = Qb.T @ u4 # for each j\nok_coord = np.allclose(Qb @ coords, u4) # u = sum_j coeff_j b^(j)\n# (ii) declaring an arbitrary basis orthonormal defines an inner product\nBb = rng.standard_normal((4, 4)) + 4.0 * np.eye(4) # an arbitrary basis\n\n\ndef ipb(u, v): # via coordinate vectors\n cu, cv = np.linalg.solve(Bb, u), np.linalg.solve(Bb, v)\n return cu @ cv\n\n\nv4, w4 = rng.standard_normal((2, 4))\na4 = rng.standard_normal()\nok_ax = (np.isclose(ipb(u4, v4), ipb(v4, u4)) # symmetry\n and np.isclose(ipb(a4 * u4 + w4, v4),\n a4 * ipb(u4, v4) + ipb(w4, v4)) # linearity\n and ipb(u4, u4) > 0) # positive definite\nG = np.array([[ipb(Bb[:, i], Bb[:, j]) for j in range(4)] for i in range(4)])\nok_onb = np.allclose(G, np.eye(4)) # B is orthonormal under ipb\nAb = np.linalg.inv(Bb @ Bb.T) # the weighted form\nok_wt = np.isclose(ipb(u4, v4), u4 @ Ab @ v4) # ipb = u^T A v\ncheck(\"[P-basis] orthonormal-basis coordinates; declared basis defines \"\n \"an inner product\", ok_coord and ok_ax and ok_onb and ok_wt)" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-cities]** The geometry of the entry's city figure: five European cities as vectors in R^2, drawn in an orthonormal frame that puts Helsinki on the horizontal line, with the dashed rulers hitting it at the projected lengths; a neuron's weighted input is maximized by the input aligned with its weights; and the pinned coordinates are re-checked against OpenStreetMap when the network is available." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "# (latitude, longitude) in degrees, retrieved 2026-08-15 from OpenStreetMap\n# via the Nominatim geocoder (https://nominatim.openstreetmap.org/search,\n# format=json, limit=1, queries \", \"); data (c) OpenStreetMap\n# contributors, ODbL. The optional check below re-verifies these values\n# against the live service whenever network access is available.\ncities = {\n \"Helsinki\": np.array([60.1666, 24.9435]),\n \"Paris\": np.array([48.8535, 2.3484]),\n \"Rome\": np.array([41.8933, 12.4829]),\n \"Madrid\": np.array([40.4168, -3.7035]),\n \"Reykjavik\": np.array([64.1460, -21.9422]),\n}\nQUERIES = {\"Helsinki\": \"Helsinki, Finland\", \"Paris\": \"Paris, France\",\n \"Rome\": \"Rome, Italy\", \"Madrid\": \"Madrid, Spain\",\n \"Reykjavik\": \"Reykjavik, Iceland\"}\nhel = cities[\"Helsinki\"]\nips_city = {name: float(hel @ v) for name, v in cities.items()\n if name != \"Helsinki\"}\nok_rank = max(ips_city, key=ips_city.get) == \"Reykjavik\"\n# the figure's frame: each city (x1, x2) is DRAWN at x1 u + x2 v for an\n# orthonormal pair u, v chosen such that Helsinki lies on the horizontal\n# line and the other cities lie above it\nalpha = np.arctan2(hel[1], hel[0])\nu_fr = np.array([np.cos(alpha), np.sin(alpha)])\nv_fr = np.array([np.sin(alpha), -np.cos(alpha)])\nok_onf = (np.isclose(u_fr @ v_fr, 0.0) # orthonormal pair\n and np.isclose(u_fr @ u_fr, 1.0) and np.isclose(v_fr @ v_fr, 1.0))\npos = {n: c[0] * u_fr + c[1] * v_fr for n, c in cities.items()}\npos_fig = {\"Helsinki\": (65.13, 0.00), \"Paris\": (46.03, 16.54),\n \"Rome\": (43.48, 4.51), \"Madrid\": (35.92, 18.90),\n \"Reykjavik\": (50.85, 44.84)}\nok_pos = all(np.allclose(pos[n], pos_fig[n], atol=0.01) for n in pos_fig)\nok_horiz = np.isclose(pos[\"Helsinki\"][1], 0.0) and pos[\"Helsinki\"][0] > 0\nok_above = all(pos[n][1] > 0 for n in cities if n != \"Helsinki\")\n# orthonormality preserves inner products: drawn positions reproduce the\n# lat/lon inner products with Helsinki exactly\nok_pres = all(np.isclose(float(pos[n] @ pos[\"Helsinki\"]), ips_city[n])\n for n in ips_city)\n# vertical rulers: each foot sits on the horizontal axis at the projected\n# length / ||hel||, so left-to-right order = inner-product order\nok_feet = all(np.isclose(pos[n][0], ips_city[n] / np.linalg.norm(hel))\n for n in ips_city)\nok_order = (sorted(ips_city, key=ips_city.get)\n == sorted(ips_city, key=lambda n: pos[n][0]))\ncheck(\"[P-cities] orthonormal frame puts Helsinki horizontal, others \"\n \"above; positions match the figure and preserve inner products\",\n ok_onf and ok_pos and ok_horiz and ok_above and ok_pres\n and ok_feet and ok_order and ok_rank)\nprint(\" \" + \", \".join(\n f\"{n}: ={ips_city[n]:.0f}, drawn=({pos[n][0]:.2f},{pos[n][1]:.2f})\"\n for n in ips_city))\n\nw_t = np.array([2.0, 1.2]) # the neuron's weight vector\nangles_n = np.linspace(0.0, 2.0 * np.pi, 720, endpoint=False)\ninputs = np.c_[np.cos(angles_n), np.sin(angles_n)] # unit-norm inputs\nbest_in = inputs[np.argmax(inputs @ w_t)]\nang_n = np.degrees(np.arccos(np.clip(\n best_in @ (w_t / np.linalg.norm(w_t)), -1.0, 1.0)))\ncheck(\"[P-cities] weighted input maximized by the template \"\n \"direction\", ang_n < 0.5)\n\n# optional: re-verify the pinned coordinates against the live authoritative\n# source (OpenStreetMap Nominatim); skipped gracefully without network\ntry:\n import json\n import time\n import urllib.parse\n import urllib.request\n\n fetched = {}\n for name, q in QUERIES.items():\n url = (\"https://nominatim.openstreetmap.org/search\"\n \"?format=json&limit=1&q=\" + urllib.parse.quote(q))\n req = urllib.request.Request(url, headers={\n \"User-Agent\": \"dictionaryappliedml-demo/1.0 (alex.jung@aalto.fi)\"})\n with urllib.request.urlopen(req, timeout=15) as resp:\n hit = json.load(resp)[0]\n fetched[name] = np.array([float(hit[\"lat\"]), float(hit[\"lon\"])])\n time.sleep(1.1) # Nominatim rate limit: max 1 req/s\n # 0.05 degrees is about five kilometers. A city has no location more\n # precise than that, and the geocoder moves its representative point now\n # and then: between the drafting of this demo and September 2026 its\n # point for Paris moved two kilometers, which broke a 0.01-degree check.\n # The pinned values stay as they are, so that the figure and the CSV do\n # not move with the geocoder; the check asks only that they still name\n # these cities.\n ok_src = all(np.allclose(cities[n], fetched[n], atol=0.05)\n for n in cities)\n check(\"[P-cities] pinned (lat, lon) values match OpenStreetMap Nominatim\",\n ok_src)\nexcept Exception as exc: # no network: keep the demo runnable\n print(f\" [--] [P-cities] skipped (no network / service unavailable: \"\n f\"{type(exc).__name__})\")" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-kernel]** The Gram matrix of a Gaussian kernel on 40 points is positive semi-definite: kernel evaluations are inner products between transformed feature vectors." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "P = rng.standard_normal((40, 2))\nd2 = ((P[:, None, :] - P[None, :, :]) ** 2).sum(-1)\nK = np.exp(-d2 / 2.0)\neigmin = float(np.linalg.eigvalsh(K).min())\ncheck(\"[P-kernel] Gaussian-kernel Gram matrix is psd\", eigmin > -1e-10)" }, { "cell_type": "markdown", "metadata": {}, "source": "**[P-dist]** The squared Euclidean distance expands into three inner products, which is why distance-based methods touch the feature vectors only through inner products." }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": "ok_dist = True\nfor _ in range(1000):\n x, yv = rng.standard_normal((2, 5))\n ok_dist &= np.isclose(np.linalg.norm(x - yv) ** 2,\n x @ x - 2 * (x @ yv) + yv @ yv)\ncheck(\"[P-dist] ||x - y||^2 = - 2 + \", ok_dist)\n\n# ---- preview figure (runs last; uses cos_prods and ips from [P-cos])\nfig, ax = plt.subplots(figsize=(3.8, 3.8))\nax.plot(cos_prods, ips, \"k.\", ms=2)\nlim = [min(ips), max(ips)]\nax.plot(lim, lim, \"k--\", lw=0.8)\nax.set_xlabel(r\"$\\|x\\|\\,\\|x'\\|\\cos\\theta$\")\nax.set_ylabel(r\"$\\langle x, x'\\rangle$\")\nax.set_title(\"angle form equals the inner product\")\nax.set_aspect(\"equal\")\nfig.tight_layout()\nfig.savefig(OUT_DIR / \"innerproduct.png\", dpi=110)\n\nn_ok = sum(ok for _, ok in report)\nprint(f\"\\n{n_ok}/{len(report)} checks pass\")\nprint(\"wrote innerproduct.png\")\nif n_ok != len(report):\n raise SystemExit(1)" } ] }