{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3",
   "language": "python"
  },
  "language_info": {
   "name": "python"
  },
  "colab": {
   "name": "transformer.ipynb"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# transformer \u2014 Python demo\n\nNumerical companion to the entry [transformer](https://dictionaryofml.org/terms/transformer.html) of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org/): it recomputes what the entry states and prints one line per check.\n\nDownloads the INCA 1-hour precipitation analysis (GeoSphere Austria) for a fixed timestamp, resamples it onto a 96 x 96 pixel image of 1 km resolution centered on Krems an der Donau, and cuts the image into 16 x 16-pixel patches. Each patch is one token of the image data point; its pixel values form the feature vector of the token, and arranging these feature vectors as rows yields the input matrix of a transformer. The course of the Danube (extracted once from OpenStreetMap, embedded below) and the location of Krems are drawn for orientation.\n\nRequires NumPy and Matplotlib only, and uses fixed seeds, so the printed numbers reproduce exactly. Generated from [`pythondemos/transformer.py`](https://dictionaryofml.org/terms/transformer.py); CC BY 4.0."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Notebook shim: the script resolves output paths relative to __file__,\n# which a notebook kernel does not define; everything lands in the\n# working directory instead.\nimport os\n__file__ = os.path.join(os.getcwd(), \"transformer.py\")\nos.makedirs(\"pythondemos\", exist_ok=True)"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "\"\"\"Radar-based precipitation image around Krems, cut into patches (tokens).\n\nDownloads the INCA 1-hour precipitation analysis (GeoSphere Austria) for a\nfixed timestamp, resamples it onto a 96 x 96 pixel image of 1 km resolution\ncentered on Krems an der Donau, and cuts the image into 16 x 16-pixel\npatches. Each patch is one token of the image data point; its pixel values\nform the feature vector of the token, and arranging these feature vectors\nas rows yields the input matrix of a transformer. The course of the Danube\n(extracted once from OpenStreetMap, embedded below) and the location of\nKrems are drawn for orientation.\n\nThe timestamp is pinned, so re-running the script downloads the same\nhistorical reading and reproduces the committed CSVs byte for byte\n(network access required).\n\nBlocks:\n    [B-fetch]   download the INCA precipitation grid around Krems\n    [B-grid]    resample onto the 96 x 96 pixel image; write transformer_radar.csv\n    [B-river]   write the Danube course in pixel coordinates; transformer_danube.csv\n    [B-preview] plot the image cut into patches; save transformer.png\n\"\"\"\nimport json\nimport urllib.request\nfrom pathlib import Path\n\nimport numpy as np\nimport matplotlib.pyplot as plt\n\nHERE = Path(__file__).parent\n\n# pinned reading: a convective-rain hour over Krems an der Donau, Austria\nTIMESTAMP = \"2026-08-20T17:00\"\nKREMS_LAT, KREMS_LON = 48.4167, 15.6167\nNPIX = 96        # image is NPIX x NPIX pixels of 1 km\nPATCH = 16       # patches are PATCH x PATCH pixels\n\n# course of the Danube near Krems as (km east, km north) offsets;\n# extracted from OpenStreetMap (waterway=river, name=Donau) on 2026-09-03\nDANUBE_KM = [\n    (-49.9, -20.7), (-49.3, -21.7), (-48.8, -22.9), (-48.0, -23.8), (-46.8, -24.3),\n    (-45.6, -24.5), (-44.3, -24.7), (-42.8, -24.4), (-41.6, -24.3), (-40.5, -24.7),\n    (-39.5, -25.8), (-38.6, -26.9), (-37.3, -27.1), (-36.8, -25.9), (-37.4, -24.7),\n    (-36.8, -23.4), (-35.7, -22.8), (-34.4, -22.6), (-33.0, -22.4), (-31.7, -22.2),\n    (-30.3, -22.3), (-28.9, -22.3), (-27.7, -21.9), (-26.5, -21.5), (-25.1, -21.3),\n    (-23.8, -21.2), (-22.3, -20.9), (-21.0, -19.8), (-19.9, -19.2), (-18.7, -18.2),\n    (-17.6, -16.4), (-16.5, -15.3), (-15.7, -14.3), (-15.4, -12.9), (-15.3, -11.7),\n    (-15.2, -10.5), (-15.5, -9.1), (-15.4, -7.7), (-15.0, -6.4), (-13.8, -5.5),\n    (-12.6, -4.9), (-11.7, -4.0), (-11.0, -3.0), (-10.2, -2.0), (-9.3, -1.2),\n    (-8.0, -1.1), (-7.5, -2.1), (-6.9, -3.3), (-5.5, -3.5), (-4.2, -2.8),\n    (-3.1, -2.1), (-1.7, -1.7), (-0.1, -1.5), (1.3, -1.6), (3.1, -3.3),\n    (4.2, -3.9), (5.9, -3.2), (7.4, -3.3), (8.4, -4.1), (9.4, -4.9),\n    (10.9, -4.8), (12.5, -4.1), (14.5, -4.1), (16.0, -3.9), (17.3, -4.1),\n    (18.5, -4.8), (19.6, -6.1), (21.7, -7.4), (22.9, -7.8), (24.0, -8.5),\n    (26.1, -8.5), (28.0, -8.2), (29.5, -8.8), (32.7, -9.0), (35.9, -9.2),\n    (39.5, -8.7), (42.8, -7.8), (44.4, -7.3), (45.7, -7.2), (46.9, -7.1),\n]"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-fetch]**"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "KM_PER_DEG_LAT = 110.57\nKM_PER_DEG_LON = 111.32 * np.cos(np.deg2rad(KREMS_LAT))\nhalf_lat = (NPIX / 2 + 2) / KM_PER_DEG_LAT\nhalf_lon = (NPIX / 2 + 2) / KM_PER_DEG_LON\nurl = (\n    \"https://dataset.api.hub.geosphere.at/v1/grid/historical/inca-v1-1h-1km\"\n    f\"?parameters=RR&start={TIMESTAMP}&end={TIMESTAMP}\"\n    f\"&bbox={KREMS_LAT - half_lat:.4f},{KREMS_LON - half_lon:.4f},\"\n    f\"{KREMS_LAT + half_lat:.4f},{KREMS_LON + half_lon:.4f}\"\n    \"&output_format=geojson\"\n)\nwith urllib.request.urlopen(url, timeout=120) as resp:\n    payload = json.load(resp)\npoints = np.array(\n    [\n        f[\"geometry\"][\"coordinates\"]\n        + [f[\"properties\"][\"parameters\"][\"RR\"][\"data\"][0]]\n        for f in payload[\"features\"]\n    ]\n)\nprint(f\"[B-fetch] {len(points)} grid cells around Krems at {TIMESTAMP} UTC\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-grid]**"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# pixel (px, py) is centered (px - 47.5, py - 47.5) km east/north of Krems\neast_km = (points[:, 0] - KREMS_LON) * KM_PER_DEG_LON\nnorth_km = (points[:, 1] - KREMS_LAT) * KM_PER_DEG_LAT\nimage = np.full((NPIX, NPIX), np.nan)\ncounts = np.zeros((NPIX, NPIX))\npx = np.rint(east_km + (NPIX - 1) / 2).astype(int)\npy = np.rint(north_km + (NPIX - 1) / 2).astype(int)\ninside = (px >= 0) & (px < NPIX) & (py >= 0) & (py < NPIX)\nfor x, y, val in zip(px[inside], py[inside], points[inside, 2]):\n    image[y, x] = (0.0 if counts[y, x] == 0 else image[y, x]) + val\n    counts[y, x] += 1\nimage[counts > 0] /= counts[counts > 0]\n# fill pixels that received no grid cell from the closest filled pixel\nmissing = np.argwhere(counts == 0)\nfilled = np.argwhere(counts > 0)\nfor y, x in missing:\n    nearest = filled[np.argmin(((filled - [y, x]) ** 2).sum(axis=1))]\n    image[y, x] = image[nearest[0], nearest[1]]\nrows = [\n    f\"{x},{y},{image[y, x]:.3f}\" for y in range(NPIX) for x in range(NPIX)\n]\ncsv_path = HERE / \"transformer_radar.csv\"\ncsv_path.write_text(\"px,py,rr\\n\" + \"\\n\".join(rows) + \"\\n\")\nprint(\n    f\"[B-grid] wrote {csv_path.name}: {NPIX}x{NPIX} pixels, \"\n    f\"{len(missing)} empty pixels filled from the closest filled pixel, \"\n    f\"max {np.nanmax(image):.2f} mm\"\n)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-river]**"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "offset = (NPIX - 1) / 2\nriver_rows = [\n    f\"{e + offset:.1f},{n + offset:.1f}\"\n    for e, n in DANUBE_KM\n    if abs(e) <= NPIX / 2 + 0.5 and abs(n) <= NPIX / 2 + 0.5\n]\nriver_path = HERE / \"transformer_danube.csv\"\nriver_path.write_text(\"px,py\\n\" + \"\\n\".join(river_rows) + \"\\n\")\nprint(f\"[B-river] wrote {river_path.name}: {len(river_rows)} points\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**[B-preview]**"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "fig, ax = plt.subplots(figsize=(6.4, 4.8))\nhalf = NPIX / 2\nim = ax.imshow(\n    image,\n    origin=\"lower\",\n    cmap=\"gray_r\",\n    vmin=0.0,\n    extent=(-half, half, -half, half),\n)\nfor line_km in range(-int(half) + PATCH, int(half), PATCH):\n    ax.axhline(line_km, color=\"0.55\", linewidth=0.5)\n    ax.axvline(line_km, color=\"0.55\", linewidth=0.5)\nriver = np.array(DANUBE_KM)\nax.plot(river[:, 0], river[:, 1], color=\"white\", linewidth=2.4)\nax.plot(river[:, 0], river[:, 1], color=\"black\", linewidth=1.0)\nax.plot(0, 0, marker=\"o\", color=\"black\", markersize=5,\n        markeredgecolor=\"white\")\nax.annotate(\"Krems\", (0, 0), xytext=(2, 3), fontsize=9,\n            bbox=dict(facecolor=\"white\", edgecolor=\"none\", pad=0.5))\nax.annotate(\"Danube\", (-32, -22.3), xytext=(-30, -19), fontsize=9,\n            bbox=dict(facecolor=\"white\", edgecolor=\"none\", pad=0.5))\nax.set_xlim(-half, half)\nax.set_ylim(-half, half)\nax.set_xlabel(\"km east of Krems\")\nax.set_ylabel(\"km north of Krems\")\nax.set_title(\n    f\"1-hour precipitation around Krems, {TIMESTAMP} UTC\\n\"\n    \"(INCA, GeoSphere Austria); each 16 x 16-pixel patch is one token\"\n)\nfig.colorbar(im, ax=ax, label=\"precipitation (mm)\")\nfig.tight_layout()\npng_path = HERE / \"transformer.png\"\nfig.savefig(png_path, dpi=150)\nprint(f\"[B-preview] saved {png_path.name}\")"
  }
 ]
}