{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3",
   "language": "python"
  },
  "language_info": {
   "name": "python"
  },
  "colab": {
   "name": "cav.ipynb"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# concept activation vector (CAV) \u2014 Python demo\n\nNumerical companion to the entry [concept activation vector (CAV)](https://dictionaryofml.org/terms/cav.html) of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org/): it recomputes what the entry states and prints one line per check.\n\nA CAV is fitted the way the entry describes it, on real data. The data points are five consecutive days of weather at Krems an der Donau (GeoSphere Austria, station 3805, 2000-2024): the daily maximum and minimum temperature and the rain of each day, fifteen numbers, and the label says whether at least 1 mm of rain fell on the sixth day. A small deep net (15 -> 16 -> 16 -> 1) predicts that label; the first hidden layer, sixteen neurons, is the layer whose activations the CAV lives in.\n\nRequires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Generated from [`pythondemos/cav.py`](https://dictionaryofml.org/terms/cav.py); CC BY 4.0."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Notebook shim: the script resolves output paths relative to __file__,\n# which a notebook kernel does not define; everything lands in the\n# working directory instead.\nimport os\n__file__ = os.path.join(os.getcwd(), \"cav.py\")\nos.makedirs(\"pythondemos\", exist_ok=True)"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "\"\"\"\ncav.py \u2014 numerical companion to the glossary entry 'concept activation\nvector (CAV)'.\n\nPurpose\n-------\nA CAV is fitted the way the entry describes it, on real data.  The data\npoints are five consecutive days of weather at Krems an der Donau\n(GeoSphere Austria, station 3805, 2000-2024): the daily maximum and minimum\ntemperature and the rain of each day, fifteen numbers, and the label says\nwhether at least 1 mm of rain fell on the sixth day.  A small deep net\n(15 -> 16 -> 16 -> 1) predicts that label; the first hidden layer, sixteen\nneurons, is the layer whose activations the CAV lives in.\n\nThe concept is a heat wave: five days in a row with a maximum of at least\n30 degrees.  It is not a feature the net was built with, and it is\nspecified by examples: windows that are heat waves and windows that are\nnot.  A linear classifier fitted to the activations of those examples\nseparates them by a hyperplane; its normal vector is the CAV.  Testing with\nCAVs asks whether the net's rain score rises when the activations move a\nlittle along the CAV, at the windows followed by rain.  A cold wave (five\ndays with a minimum of at most -5 degrees) is fitted the same way.  Both\nconcepts turn out to be ones the net uses, in opposite directions, and the\nrecord says why: rain follows a heat wave more often than an average\nwindow (30% against 23%) and a cold wave less often (16%).  The control the method rests on is\nthe random concept, a set of windows drawn at random and labelled as if\nthey carried a concept, whose TCAV score sits near one half.\n\nThe figure draws a plane a reader can look at: through the mean activation,\nspanned by the CAV and the leading direction of the activations orthogonal\nto it.  In that plane the concept hyperplane is a vertical line and the\nCAV the horizontal axis, so the separation the classifier found is visible\nas it is; the net's decision boundary, the curve where its score on that\nplane crosses the base-rate threshold, is what the rain score does across\nit.  A first version used a two-neuron layer so that the activations\nthemselves formed a plane; there every direction was either along the\nscore's gradient or against it, random concepts scored 0 or 1 and the\ncontrol was worthless.  Sixteen neurons restore it.\n\nSelf-contained (numpy + matplotlib, urllib for the download), fixed seed.\n\nBlocks\n------\n[B-fetch]  25 years of daily tmin, tmax and rain at Krems, downloaded from\n           the GeoSphere data hub and written to cav_weather.csv; five-day\n           windows and their sixth-day rain labels.\n[B-net]    A deep net 15 -> 16 -> 16 -> 1 (ReLU, logistic output) trained\n           with Adam on 2000-2018 and tested on 2019-2024; it beats\n           always answering the base rate on the held-out years.  Rain is predicted\n           when the score exceeds the base rate.\n[B-cav]    The CAV: a binary linear classifier separating heat-wave windows\n           from non-heat-wave windows by their sixteen activations, fitted\n           to forty examples of each kind, all of which the figure draws.\n           Its weight vector, the CAV, is the normal of the separating\n           hyperplane; the same fit on every window of the record gives a\n           nearby direction.\n[B-tcav]   Conceptual sensitivity: the directional derivative of the rain\n           score along the unit CAV at the test windows followed by rain,\n           and the TCAV score, the fraction of them where it is positive.\n           The CAV is refitted against fresh draws of non-concept windows,\n           and random \"concepts\" (windows drawn at random as positives)\n           are scored the same way: their scores are coin flips between 0\n           and 1 with mean one half, since the score's gradient points the\n           same way at nearly every rainy window, and a concept counts\n           when its refits agree (the original method's two-sided test).\n           The cold wave is scored the same way.\n[B-fig]    The figure: the plane through the mean activation spanned by\n           the CAV and the leading orthogonal direction; the test windows,\n           the eighty examples and the concept hyperplane projected onto\n           it, and the net's boundary on it.  Written to\n           cav_points.csv, cav_windows.csv, cav_boundary.csv,\n           cav_cavline.csv, cav_arrow.csv and cav.png.\n\nOutputs\n-------\ncav_weather.csv  : date, tmin, tmax, rr for every day downloaded\ncav_points.csv   : z1, z2, concept, grp for the eighty concept examples\n                   (plane coordinates: z1 along the CAV)\ncav_windows.csv  : z1, z2, rain for a sample of test windows\ncav_boundary.csv : the net's decision boundary on the plane\ncav_cavline.csv  : the concept hyperplane\ncav_arrow.csv    : the CAV, drawn from a point on the hyperplane\ncav.png          : preview (checking only)\n\"\"\"\n\nimport json\nimport urllib.request\nfrom pathlib import Path\n\nimport numpy as np\nimport matplotlib\n\nOUT_DIR = Path(__file__).parent\n\nmatplotlib.use(\"Agg\")\nimport matplotlib.pyplot as plt\n\nreport = []                         # collects (check name, pass/fail) pairs\n\n\ndef check(name, ok):                # records and prints one verification\n    report.append((name, bool(ok)))\n    print(f\"  [{'ok' if ok else 'FAIL'}] {name}\")\n\n\nrng = np.random.default_rng(20260917)\nDAYS = 5                            # a data point is this many days in a row\nHOT, COLD = 30.0, -5.0              # heat wave: tmax >= HOT on all days;\n                                    # cold wave: tmin <= COLD on all days\nWET = 1.0                           # mm on the sixth day counting as rain\nNEX = 40                            # concept examples of each kind, all drawn"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-fetch]** 25 years of daily tmin, tmax and rain at Krems, downloaded from the GeoSphere data hub and written to cav_weather.csv; five-day windows and their sixth-day rain labels."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "URL = (\"https://dataset.api.hub.geosphere.at/v1/station/historical/\"\n       \"klima-v2-1d?parameters=tlmin,tlmax,rr&station_ids=3805\"\n       \"&start=2000-01-01&end=2024-12-31\")\nwith urllib.request.urlopen(URL, timeout=180) as resp:\n    payload = json.load(resp)\nparams = payload[\"features\"][0][\"properties\"][\"parameters\"]\nstamps = [t[:10] for t in payload[\"timestamps\"]]\ntmin = np.array([np.nan if v is None else v for v in params[\"tlmin\"][\"data\"]])\ntmax = np.array([np.nan if v is None else v for v in params[\"tlmax\"][\"data\"]])\nrain = np.array([np.nan if v is None else v for v in params[\"rr\"][\"data\"]])\nwith open(OUT_DIR / \"cav_weather.csv\", \"w\") as fh:\n    fh.write(\"date,tmin,tmax,rr\\n\")\n    for d, lo, hi, r in zip(stamps, tmin, tmax, rain):\n        fh.write(f\"{d},{lo},{hi},{r}\\n\")\ncheck(f\"[B-fetch] {len(stamps)} days downloaded, 2000-2024\",\n      len(stamps) > 9000 and stamps[0] == \"2000-01-01\")\n\n# five-day windows: features are the five maxima, the five minima and the\n# five rain amounts (log-scaled); the label is rain on the day after\nn = len(stamps) - DAYS\nX = np.stack([np.stack([tmax[i:i + DAYS], tmin[i:i + DAYS],\n                        np.log1p(np.maximum(rain[i:i + DAYS], 0.0))]).ravel()\n              for i in range(n)])\ny = (rain[DAYS:DAYS + n] >= WET).astype(float)\nyear = np.array([int(stamps[i + DAYS][:4]) for i in range(n)])\nok_rows = ~np.isnan(X).any(axis=1) & ~np.isnan(rain[DAYS:DAYS + n])\nX, y, year, win_start = X[ok_rows], y[ok_rows], year[ok_rows], np.arange(n)[ok_rows]\nheat = (X[:, :DAYS] >= HOT).all(axis=1)\ncold = (X[:, DAYS:2 * DAYS] <= COLD).all(axis=1)\nprint(f\"  {len(y)} windows of {DAYS} days; rain follows {y.mean():.0%} of them; \"\n      f\"{heat.sum()} heat waves, {cold.sum()} cold waves\")\ncheck(\"[B-fetch] both concepts occur often enough to supply examples\",\n      heat.sum() >= 2 * NEX and cold.sum() >= 2 * NEX)\n\ntr, te = year <= 2018, year >= 2019\nmu, sd = X[tr].mean(axis=0), X[tr].std(axis=0)\nXs = (X - mu) / sd"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-net]** A deep net 15 -> 16 -> 16 -> 1 (ReLU, logistic output) trained with Adam on 2000-2018 and tested on 2019-2024; it beats always answering the base rate on the held-out years. Rain is predicted when the score exceeds the base rate."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "H1, H2 = 16, 16\n\n\nrelu = lambda a: np.maximum(a, 0.0)\n\n\ndef forward(Xin, P):\n    W1, b1, W2, b2, w3, c = P\n    Z = relu(Xin @ W1 + b1)                         # the chosen layer (n, 16)\n    Hh = relu(Z @ W2 + b2)\n    return Z, Hh, Hh @ w3 + c                       # score: larger, rain likelier\n\n\ndef score_from_plane(Z, P):\n    \"\"\"The rest of the net, s(z): from the chosen layer's activations to the\n    score.\"\"\"\n    _, _, W2, b2, w3, c = P\n    return relu(Z @ W2 + b2) @ w3 + c\n\n\nP = [rng.normal(0, np.sqrt(2 / X.shape[1]), (X.shape[1], H1)), np.zeros(H1),\n     rng.normal(0, np.sqrt(2 / H1), (H1, H2)), np.zeros(H2),\n     rng.normal(0, 0.3, H2), 0.0]\nWD = 3e-3                                           # weight decay\n# Adam (Kingma and Ba, 2015): plain gradient descent with a small step\n# left this net answering the base rate everywhere\nm1 = [np.zeros_like(np.asarray(q, dtype=float)) for q in P]\nm2 = [np.zeros_like(np.asarray(q, dtype=float)) for q in P]\nlrate, losses = 0.01, []\nXtr, ytr = Xs[tr], y[tr]\nfor step in range(1500):\n    Z, Hh, s = forward(Xtr, P)\n    p = 1.0 / (1.0 + np.exp(-s))\n    losses.append(float(-np.mean(ytr * np.log(p + 1e-9)\n                                 + (1 - ytr) * np.log(1 - p + 1e-9))))\n    err = (p - ytr) / len(ytr)                      # d loss / d score\n    W1, b1, W2, b2, w3, c = P\n    g_w3, g_c = Hh.T @ err + WD * w3, err.sum()\n    dH = np.outer(err, w3) * (Hh > 0)\n    g_W2, g_b2 = Z.T @ dH + WD * W2, dH.sum(axis=0)\n    dZ = (dH @ W2.T) * (Z > 0)\n    g_W1, g_b1 = Xtr.T @ dZ + WD * W1, dZ.sum(axis=0)\n    grads = [g_W1, g_b1, g_W2, g_b2, g_w3, g_c]\n    newP = []\n    for k, (q, g) in enumerate(zip(P, grads)):\n        m1[k] = 0.9 * m1[k] + 0.1 * g\n        m2[k] = 0.999 * m2[k] + 0.001 * g * g\n        mh = m1[k] / (1 - 0.9 ** (step + 1))\n        vh = m2[k] / (1 - 0.999 ** (step + 1))\n        newP.append(q - lrate * mh / (np.sqrt(vh) + 1e-8))\n    P = newP\n\nZall, _, sall = forward(Xs, P)\npall = 1.0 / (1.0 + np.exp(-sall))\n# the net's decision rule: predict rain when the score exceeds the base\n# rate of the years it was fitted to (a score above THR)\nTHR = float(np.log(y[tr].mean() / (1 - y[tr].mean())))\nll = lambda m: float(-np.mean(y[m] * np.log(pall[m] + 1e-9)\n                              + (1 - y[m]) * np.log(1 - pall[m] + 1e-9)))\nll_const = float(-np.mean(y[te] * np.log(y[tr].mean())\n                          + (1 - y[te]) * np.log(1 - y[tr].mean())))  # the base-rate rule\nacc_te = float(((sall[te] > THR) == (y[te] > 0.5)).mean())\nhit = float((sall[te][y[te] > 0.5] > THR).mean())\nprint(f\"  loss {losses[0]:.3f} -> {losses[-1]:.3f}; test log-loss {ll(te):.3f} \"\n      f\"against {ll_const:.3f} for always answering the base rate; at the \"\n      f\"base-rate threshold the net predicts rain for {hit:.0%} of the rainy \"\n      f\"test days and is right on {acc_te:.0%} of all test days\")\ncheck(\"[B-net] the net beats always answering the base rate on the held-out years\",\n      ll(te) < ll_const - 0.01)\n\n\ndef fit_linear(Xin, yin, steps=2000, lr=0.3, l2=1e-2):\n    \"\"\"Logistic regression by gradient descent, with a small ridge penalty\n    so that separable examples still fix a direction; returns (w, b).\"\"\"\n    w, b = np.zeros(Xin.shape[1]), 0.0\n    for _ in range(steps):\n        p = 1.0 / (1.0 + np.exp(-(Xin @ w + b)))\n        w -= lr * (Xin.T @ (p - yin) / len(yin) + l2 * w)\n        b -= lr * (p - yin).mean()\n    return w, b\n\n\nacc = lambda w, b, Xin, yin: float((((Xin @ w + b) > 0) == (yin > 0.5)).mean())\n\n# what the record itself says about the two concepts\nafter_heat, after_cold = float(y[heat].mean()), float(y[cold].mean())\nprint(f\"  rain follows {after_heat:.0%} of the heat waves and {after_cold:.0%} \"\n      f\"of the cold waves, against {y.mean():.0%} of all windows\")\ncheck(\"[B-net] the record says rain follows a heat wave more often than \"\n      \"an average window\", after_heat > y.mean() + 0.05)\ncheck(\"[B-net] ... and a cold wave less often\", after_cold < y.mean() - 0.05)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-cav]** The CAV: a binary linear classifier separating heat-wave windows from non-heat-wave windows by their sixteen activations, fitted to forty examples of each kind, all of which the figure draws. Its weight vector, the CAV, is the normal of the separating hyperplane; the same fit on every window of the record gives a nearby direction."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "def examples(concept, n=NEX):\n    \"\"\"n concept windows and n non-concept windows, drawn at random from\n    the record: the examples a user would supply, not a curated selection.\"\"\"\n    pos = rng.choice(np.where(concept)[0], n, replace=False)\n    neg = rng.choice(np.where(~concept)[0], n, replace=False)\n    idx = np.concatenate([pos, neg])\n    return idx, np.concatenate([np.ones(n), np.zeros(n)])\n\n\nidx_ex, y_ex = examples(heat)\nZ_ex = Zall[idx_ex]\nw_cav, b_cav = fit_linear(Z_ex, y_ex)\ncav = w_cav / np.linalg.norm(w_cav)                 # the CAV, unit length\nprint(f\"  the CAV is fitted to {len(idx_ex)} examples, {int(y_ex.sum())} of \"\n      f\"them heat waves; it points along ({cav[0]:.2f}, {cav[1]:.2f})\")\nn_sep = int(round(acc(w_cav, b_cav, Z_ex, y_ex) * len(y_ex)))\nprint(f\"  the fitted hyperplane separates {n_sep} of the {len(y_ex)} examples\")\ncheck(\"[B-cav] the fitted hyperplane separates nearly all eighty examples\",\n      n_sep >= 0.85 * len(y_ex))\n# ten examples of each kind fix the direction: the same fit on every window\n# of the record is the reference\nw_pool, _ = fit_linear(Zall, heat.astype(float))\ncos_pool = float(cav @ (w_pool / np.linalg.norm(w_pool)))\nprint(f\"  the same fit on all {len(y)} windows gives a direction \"\n      f\"{np.degrees(np.arccos(min(cos_pool, 1.0))):.0f} degrees away\")\ncheck(\"[B-cav] forty examples of each kind fix a direction near the record's\",\n      cos_pool > 0.8)\n\nidx_cold, y_cold = examples(cold)\nw_cc, b_cc = fit_linear(Zall[idx_cold], y_cold)\ncav_cold = w_cc / np.linalg.norm(w_cc)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-tcav]** Conceptual sensitivity: the directional derivative of the rain score along the unit CAV at the test windows followed by rain, and the TCAV score, the fraction of them where it is positive. The CAV is refitted against fresh draws of non-concept windows, and random \"concepts\" (windows drawn at random as positives) are scored the same way: their scores are coin flips between 0 and 1 with mean one half, since the score's gradient points the same way at nearly every rainy window, and a concept counts when its refits agree (the original method's two-sided test). The cold wave is scored the same way."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "EPS = 1e-4\n\n\ndef sensitivity(direction, Zpts):\n    \"\"\"Directional derivative of the rain score along the unit direction.\"\"\"\n    d = direction / np.linalg.norm(direction)\n    return (score_from_plane(Zpts + EPS * d, P) - score_from_plane(Zpts, P)) / EPS\n\n\nrainy = te & (y > 0.5)                              # test windows followed by rain\nZr = Zall[rainy]\ntcav = lambda v: float((sensitivity(v, Zr) > 0).mean())\ntcav_heat, tcav_cold = tcav(cav), tcav(cav_cold)\nprint(f\"  moving along the heat-wave CAV raises the rain score at \"\n      f\"{tcav_heat:.0%} of the {len(Zr)} rainy test windows, along the \"\n      f\"cold-wave CAV at {tcav_cold:.0%}\")\n\n# does the choice of non-concept examples move the score?  and the random-\n# concept reference (the original method's statistical test)\nREFITS = 100\nscores_heat, scores_rand = [], []\nfor _ in range(REFITS):\n    i, yy = examples(heat)\n    w, _ = fit_linear(Zall[i], yy)\n    scores_heat.append(tcav(w))\n    i = rng.choice(len(y), 2 * NEX, replace=False)   # a random \"concept\"\n    w, _ = fit_linear(Zall[i], yy)\n    scores_rand.append(tcav(w))\nscores_heat, scores_rand = np.array(scores_heat), np.array(scores_rand)\nprint(f\"  over {REFITS} refits with fresh non-concept draws the heat-wave TCAV \"\n      f\"score is {scores_heat.mean():.2f} +- {scores_heat.std():.2f}; random \"\n      f\"concepts give {scores_rand.mean():.2f} +- {scores_rand.std():.2f}\")\n# The score's gradient points the same way at nearly every rainy window\n# (the net is close to one-dimensional in its activations), so a direction\n# either raises the score at almost all of them or at almost none: the\n# scores of random concepts are coin flips between 0 and 1 with mean one\n# half, and a concept counts when its refits agree, as the two-sided test\n# of the original method asks\nagree = float((scores_heat > 0.5).mean())\nt_stat = ((scores_heat.mean() - scores_rand.mean())\n          / np.sqrt(scores_heat.var() / REFITS + scores_rand.var() / REFITS))\nprint(f\"  {agree:.0%} of the heat-wave refits score above one half; \"\n      f\"Welch t-statistic against the random concepts {t_stat:.1f}\")\ncheck(\"[B-tcav] the heat-wave TCAV score is far above one half\",\n      scores_heat.mean() > 0.75)\ncheck(\"[B-tcav] the cold-wave score lies on the other side of one half\",\n      tcav_cold < 0.25)\ncheck(\"[B-tcav] random concepts score one half on average\",\n      abs(scores_rand.mean() - 0.5) < 0.15)\ncheck(\"[B-tcav] the heat-wave refits agree, unlike the coin flips\",\n      agree >= 0.8 and t_stat > 4)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-fig]** The figure: the plane through the mean activation spanned by the CAV and the leading orthogonal direction; the test windows, the eighty examples and the concept hyperplane projected onto it, and the net's boundary on it. Written to cav_points.csv, cav_windows.csv, cav_boundary.csv, cav_cavline.csv, cav_arrow.csv and cav.png."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# The plane: through the mean activation, spanned by the unit CAV (first\n# axis) and the leading principal direction of the activations after their\n# CAV component is removed (second axis). Plane coordinates of a point z:\n# a = cav . (z - zbar), b = u2 . (z - zbar).\nzbar = Zall[te].mean(axis=0)\nresid = (Zall[te] - zbar) - np.outer((Zall[te] - zbar) @ cav, cav)\n_, _, vt = np.linalg.svd(resid, full_matrices=False)\nu2 = vt[0]\nto_plane = lambda Zin: np.stack([(Zin - zbar) @ cav, (Zin - zbar) @ u2], 1)\nfrom_plane = lambda a, b: zbar + np.outer(a, cav) + np.outer(b, u2)\n\nQ_ex = to_plane(Z_ex)\nwith open(OUT_DIR / \"cav_points.csv\", \"w\") as fh:\n    fh.write(\"z1,z2,concept,grp\\n\")\n    for (a, b), k in zip(Q_ex, y_ex):\n        fh.write(f\"{a:.4f},{b:.4f},{int(k)},k{int(k)}\\n\")\nsample = rng.choice(np.where(te)[0], 400, replace=False)\nQ_win = to_plane(Zall[sample])\nwith open(OUT_DIR / \"cav_windows.csv\", \"w\") as fh:\n    fh.write(\"z1,z2,rain\\n\")\n    for (a, b), i in zip(Q_win, sample):\n        fh.write(f\"{a:.4f},{b:.4f},{int(y[i])}\\n\")\nlo, hi = Q_win.min(axis=0) - 0.2, Q_win.max(axis=0) + 0.2\n\n# the net's boundary on the plane: for each a, the b where the score on the\n# plane crosses the base-rate threshold\ngrid_a, grid_b = np.linspace(lo[0], hi[0], 161), np.linspace(lo[1], hi[1], 801)\nwith open(OUT_DIR / \"cav_boundary.csv\", \"w\") as fh:\n    fh.write(\"z1,z2\\n\")\n    for a in grid_a:\n        sc = score_from_plane(from_plane(np.full_like(grid_b, a), grid_b), P) - THR\n        sign = np.where(np.diff(np.sign(sc)))[0]\n        if len(sign):\n            fh.write(f\"{a:.4f},{grid_b[sign[0]]:.4f}\\n\")\nbnd = np.loadtxt(OUT_DIR / \"cav_boundary.csv\", delimiter=\",\", skiprows=1, ndmin=2)\ncheck(\"[B-fig] the net's boundary crosses the plane\", len(bnd) > 20)\n\n# the concept hyperplane w . z + b = 0 meets the plane in the vertical line\n# a = a0 (its normal IS the first axis); the CAV is drawn from that line\na0 = float(-(b_cav + w_cav @ zbar) / np.linalg.norm(w_cav))\nwith open(OUT_DIR / \"cav_cavline.csv\", \"w\") as fh:\n    fh.write(\"z1,z2\\n\")\n    fh.write(f\"{a0:.4f},{lo[1]:.4f}\\n{a0:.4f},{hi[1]:.4f}\\n\")\nfoot = np.array([a0, 0.5 * (lo[1] + hi[1])])\nARROW = 0.25 * (hi[0] - lo[0])\nwith open(OUT_DIR / \"cav_arrow.csv\", \"w\") as fh:\n    fh.write(\"z1,z2\\n\")\n    fh.write(f\"{foot[0]:.4f},{foot[1]:.4f}\\n\")\n    fh.write(f\"{foot[0] + ARROW:.4f},{foot[1]:.4f}\\n\")\nprint(\"  wrote cav_points.csv, cav_windows.csv, cav_boundary.csv, \"\n      \"cav_cavline.csv, cav_arrow.csv\")\n\nfig, ax = plt.subplots(figsize=(5.8, 5.8))\nax.plot(Q_win[:, 0], Q_win[:, 1], \".\", ms=3, color=\"0.6\", label=\"test window\")\nax.plot(bnd[:, 0], bnd[:, 1], \"k-\", lw=2.0,\n        label=\"decision boundary of the deep net\")\nax.plot([a0, a0], [lo[1], hi[1]], \"k--\", lw=2.0,\n        label=\"concept hyperplane (linear classifier)\")\nfor k, fill, lbl in ((1, \"k\", \"heat-wave example\"),\n                     (0, \"none\", \"non-heat-wave example\")):\n    sel = y_ex == k\n    ax.plot(Q_ex[sel, 0], Q_ex[sel, 1], \"o\", ms=8, mfc=fill, mec=\"k\", lw=0,\n            label=lbl)\nax.annotate(\"\", xy=(foot[0] + ARROW, foot[1]), xytext=foot,\n            arrowprops=dict(arrowstyle=\"-|>\", lw=2.0, color=\"k\"))\nax.annotate(\"CAV\", (foot[0] + ARROW + 0.02 * (hi[0] - lo[0]), foot[1]),\n            fontsize=10, va=\"center\")\nax.set_xlim(lo[0], hi[0])\nax.set_ylim(lo[1], hi[1])\nax.set_xlabel(\"activation along the CAV\")\nax.set_ylabel(\"activation along the leading orthogonal direction\")\nax.set_title(\"a concept is a direction in the activations of a hidden layer\",\n             fontsize=10)\nax.legend(frameon=False, fontsize=7.5, loc=\"upper center\",\n          bbox_to_anchor=(0.5, -0.13), ncol=2)\nfig.tight_layout()\nfig.savefig(OUT_DIR / \"cav.png\", dpi=110)\nprint(f\"  wrote {OUT_DIR / 'cav.png'}\")\n\nbad = [n for n, ok in report if not ok]\nprint(f\"\\n{len(report) - len(bad)}/{len(report)} checks passed\"\n      + (f\"; FAILED: {bad}\" if bad else \"\"))"
  }
 ]
}