From cdd9604bc67e78659f5de80a6fd2b7497fc36499 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 12 Aug 2026 20:38:50 +0000 Subject: [PATCH 1/5] D-CZ-1 PASSES, and reproducing D-CZ-0 found it had no artifact behind it Two findings, one run (probes/weather-p1/substrate_comfort_d_cz_0_1.py). 1. D-CZ-0 was marked DONE with NO committed script or JSON. Its figures were quoted in the plan, the arc entry, the LATEST_STATE row and three PR bodies. Worse: the #945 self-audit -- run specifically because this arc keeps shipping unbacked summary claims -- reported those figures as "verified" while only comparing the arc entry to the PLAN. Both are prose. A figure cited in two documents is cited twice, not confirmed once. Reproduced now: 4 of 9 rows (ratios 1.004/1.022/0.994/0.931). The five EXCLUDED land candidates are unreproducible -- their box centres were never recorded anywhere -- and no coordinates were invented to fake them. The |grad p| definition was never committed either, so four candidates were computed and the winner decided from data: Pa per grid cell with NO cos(lat) metric (max dev 0.069; next-best 0.398). That understates high latitude by 1/cos(lat), so R3 is ~40% low; metric-corrected the ladder is 10.3/15.5/61.2/100.9 -- ORDER survives, range widens 9.3x -> 9.8x. Two defects in the reproduction caught by its own guards: measuring all 19 storms at the preflight timestep instead of each storm's own t0 (inverted R3/R4 and would have read as C1 failing), and a seam assertion firing on a real storm at 353.4 E where a plain slice gives a one-column box. 2. D-CZ-1 PASSES. Both controls lose to both real arms on both metrics in all four regimes; GEO-DEGENERATE saturates 92-97%, so the mechanism is visible rather than assumed. C1b separation = 6.28 vs the >= 3 bar. 3. ...and the same run AMENDS C4. rho is SATURATED on the diagonal -- real-arm spread 3e-6..4.7e-5 -- so C4 as pre-registered could not have fired. rho keeps four orders of magnitude of range OFF the diagonal, so L keeps it; C4 moves to RMSE in Pa with rho as a floor check. Legitimate because D-CZ-1's whole purpose is to test the apparatus before the expensive cells and no C4 cell has been scored; illegitimate the instant one has. Recorded as an amendment carrying its trigger, and propagated to the header and the C4 bar rather than left contradicting them. A HINT, explicitly not a result: CAL-ABS beats CAL-RANK on RMSE everywhere and its margin shrinks as the field gets more active (3.96 -> 1.85 -> 1.14), R4 breaking the monotone at 1.49. One timestep, no control on that comparison, and diagonal-vs-diagonal is not what the hypothesis is about. Recorded so a later run cannot present it as a confirmation. EPIPHANIES: E-A-FIGURE-CITED-TWICE-IS-NOT-CONFIRMED-ONCE-1 and E-THE-METRIC-THAT-SEPARATES-ONE-COMPARISON-IS-BLIND-TO-ANOTHER-1 (prepend, suffix-verified). STATUS_BOARD D-CZ-0/D-CZ-1 updated. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi --- .claude/board/EPIPHANIES.md | 97 ++++ .claude/board/STATUS_BOARD.md | 4 +- .claude/plans/substrate-comfort-zones-v1.md | 136 +++++- .../exec-runs/substrate_comfort_d_cz_0_1.txt | 1 + .../substrate_comfort_d_cz_0_1.json | 452 ++++++++++++++++++ .../weather-p1/substrate_comfort_d_cz_0_1.py | 444 +++++++++++++++++ 6 files changed, 1129 insertions(+), 5 deletions(-) create mode 100644 probes/weather-p1/exec-runs/substrate_comfort_d_cz_0_1.txt create mode 100644 probes/weather-p1/substrate_comfort_d_cz_0_1.json create mode 100644 probes/weather-p1/substrate_comfort_d_cz_0_1.py diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 5b95b4d11..f63cb06bc 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,100 @@ +## 2026-08-12 — E-A-FIGURE-CITED-TWICE-IS-NOT-CONFIRMED-ONCE-1 + +**Status:** FINDING `[G]` — measured. Probe +`probes/weather-p1/substrate_comfort_d_cz_0_1.py` + `.json`, one run, +committed with the artifact this entry is about. + +**The defect.** `substrate-comfort-zones-v1` §1's nine-row regime preflight +was marked **DONE** on `STATUS_BOARD.md`, quoted in the `PR_ARC_INVENTORY` +entry, quoted again in the `LATEST_STATE` row, and its headline (a 9.3× +dynamic range across four regimes) was carried into three PR bodies. **No +script and no JSON producing any of those figures had ever been committed.** + +**The part that makes it worth an entry rather than a fix.** On #945 I ran +an explicit self-audit *specifically because* this arc keeps shipping +summary claims that do not survive checking — and reported the regime-ladder +figures as verified. What that audit actually did was confirm the arc entry +matched the PLAN. Both are prose. Neither is a measurement. **A figure cited +in two documents has been cited twice, not confirmed once** — and an audit +that compares two documents to each other measures their consistency, which +is not the property anyone cares about. + +**The rule.** An audit must terminate at an ARTIFACT — a committed script, a +committed JSON, a re-run — or it is a spell-check. Concretely: when a +board row says DONE, the check is *"name the file that produced it"*, not +*"does the summary match the plan"*. + +**What the reproduction then found**, none of which was visible from the +prose: + +1. **Only 4 of 9 rows are reproducible at all.** R1/R2/R3 centres are in the + plan and R4's come from committed `comet_tail_f14.json` (the qualifying + filter — `status == "OK"` AND `displacement_km ≥ 250` — itself had to be + *recovered* by reproducing the stored `n_qualifying = 19`, and is now + asserted in code). The five EXCLUDED land candidates carry measured + figures whose **box centres were never written down anywhere**. They are + unreproducible as committed. No coordinates were invented to fake them. +2. **The `|∇p|` definition was never committed either — only values.** Four + candidates were computed and the winner decided from the data: the + recorded figures are **Pa per grid cell with NO cos(lat) metric** + (max deviation 0.069; the next-best candidate is off by 0.398). It is a + plain `np.gradient` over the raw lat/lon array. +3. **Consequence, bounded:** ignoring cos(lat) understates the ZONAL + gradient by `1/cos(lat)`, so R3 at 60 N is ~40 % low. Metric-corrected + the ladder reads 10.3 / 15.5 / 61.2 / 100.9 — the **ORDER survives** and + the dynamic range **widens** 9.3× → ≈9.8×. The regime axis stands; the + recorded magnitudes do not. Reported as a correction, not as a collapse. +4. **A defect in the reproduction itself, caught by its own guard.** The + first pass measured all 19 storm boxes at the *preflight* timestep rather + than at each storm's own `t0` — "the places where storms once were, at an + unrelated hour". That inverted R3/R4 (220 vs 208) and would have read as + the regime ladder failing C1. Fetching per storm (19 extra chunks) + restores 37 / 56 / 220 / 363. Separately, a seam assertion fired on a + real storm centred at 353.4 E: a plain longitude slice there yields a + one-column box and a plausible, meaningless number. **Both were caught + because the probe asserted rather than assumed.** + +--- + +## 2026-08-12 — E-THE-METRIC-THAT-SEPARATES-ONE-COMPARISON-IS-BLIND-TO-ANOTHER-1 + +**Status:** FINDING `[G]` — measured, same run. + +**`ρ` is saturated on the diagonal and enormous off it.** Measured over four +regimes at 256 palette levels on MSLP: + +| comparison | `ρ` separation | +|---|---| +| real arm vs real arm (`CAL-ABS` vs `CAL-RANK`, both own-calibration) | **3×10⁻⁶ … 4.7×10⁻⁵** | +| real vs degraded (`CAL-SHUFFLE` 0.003–0.159, `GEO-DEGENERATE` 0.29–0.48) | **~0.5 … ~1.0** | + +Both real arms reconstruct the ordering essentially perfectly (ρ > 0.99996), +because 256 levels on a smooth pressure field is a very fine quantization. +So `ρ` has **four orders of magnitude of range for transfer loss `L`** (an +off-diagonal quantity) and **none at all for C4** (which compares two real +arms on their own diagonals). RMSE, which the same rewrite had just demoted, +*does* separate the real arms: ratios 3.96 / 1.85 / 1.14 / 1.49. + +**The rule: a metric is not good or bad, it is good or bad FOR A GIVEN +CONTRAST.** Choosing one for a plan as a whole — as the cross-swap rewrite +did, elevating `ρ` and demoting RMSE in one move — is a category error the +moment the plan contains two contrasts with different dynamic ranges. The +resolution here is not to revert: `L` keeps `ρ`, C4 moves to RMSE in Pa with +`ρ` as a floor check (< 0.999 = broken, not merely lost). + +**Why amending after data is legitimate here, and when it stops being.** +D-CZ-1's *stated purpose* is to test the apparatus before the expensive +cells, and **no C4 cell has been scored** — a metric found blind in +preflight is exactly what preflight is for. It becomes illegitimate the +instant one C4 measurement exists. Recorded as an amendment carrying its +trigger (`§6.4`), never edited into the bar silently. + +**Cross-ref:** this is the same shape as #926's Fisher-z result (8.3× better +in the storm tail on the raw field, 4.7× worse on ring means — one encoding, +opposite verdicts by what it represents), which §0 already cites as a +founding motivation for this plan. The plan's own primary metric turned out +to be another instance of the phenomenon the plan was built to map. + ## 2026-08-12 — E-A-HORSE-RACE-IS-NOT-A-CROSS-SWAP-1 **Status:** FINDING `[H]` — methodological, operator-ruled. Caught on diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 5fa7e4e0e..74b3139ce 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -7,8 +7,8 @@ already run and it corrected two regime definitions before any bar existed. | D-id | Deliverable | Status | Feeds | |---|---|---|---| -| D-CZ-0 | §1 regime preflight (`\|∇p\|` ladder, elevation-confound screen, speed-is-not-the-discriminator finding) | **DONE** — ladder R1 Amazon 10.2 → R2 ocean 14.9 → R3 W Siberia 43.8 → R4 storm 95.6 (9.3× range); 4 land candidates excluded on elev σ > 150 m | the regime axis all other rows score on | -| D-CZ-1 | C0 controls (shuffled codebook + degenerate geometry), losability-smoke-tested BEFORE the full run | Queued | gates every cell — a control that can't lose voids its cell | +| D-CZ-0 | §1 regime preflight (`\|∇p\|` ladder, elevation-confound screen, speed-is-not-the-discriminator finding) | **DONE — and now REPRODUCED + partly corrected (§6.1/6.2).** It had NO committed script or JSON when marked DONE. 4 rows reproduce (1.004/1.022/0.994/0.931); the 5 EXCLUDED land candidates are **unreproducible** (centres never recorded). Definition identified: Pa per grid cell **without cos(lat)** — R3 ~40 % low; metric-corrected ladder 10.3/15.5/61.2/100.9, order survives, range 9.3× → **9.8×**. Original text:** — ladder R1 Amazon 10.2 → R2 ocean 14.9 → R3 W Siberia 43.8 → R4 storm 95.6 (9.3× range); 4 land candidates excluded on elev σ > 150 m | the regime axis all other rows score on | +| D-CZ-1 | C0 controls (shuffled codebook + degenerate geometry), losability-smoke-tested BEFORE the full run | **DONE — PASS** (`substrate_comfort_d_cz_0_1.py/.json`). Both controls lose to both real arms on BOTH metrics in ALL FOUR regimes; mechanism visible (`GEO-DEGENERATE` saturates 92–97 %). C1b `separation` = **6.28** vs the ≥ 3 bar. **AND the run amended C4**: `ρ` is SATURATED on the diagonal (real-arm spread 3e-6…4.7e-5) so C4 could not have fired — `L` keeps `ρ` off-diagonal, C4 moves to RMSE in Pa (§6.4) | gates every cell — a control that can't lose voids its cell | | D-CZ-2 | C1 regime-ladder stability across ≥3 timesteps | Queued | anti-cherry-pick on the whole regime axis | | D-CZ-2b | **C1b constancy is measured** — `separation = (between-box range)/(mean within-box σ)` ≥ 3 | Queued | earns the phrase "held constant"; otherwise a caveat rides every cell | | D-CZ-2c | **C1c the suitability ASSUMPTION** — regimes must differ in autocorrelation decay + rank-distribution shape, not only in `\|∇p\|` | Queued | a null here VOIDS the cross-swap reading (spread manufactured on the wrong axis) | diff --git a/.claude/plans/substrate-comfort-zones-v1.md b/.claude/plans/substrate-comfort-zones-v1.md index ce5b6717c..755982a31 100644 --- a/.claude/plans/substrate-comfort-zones-v1.md +++ b/.claude/plans/substrate-comfort-zones-v1.md @@ -30,8 +30,17 @@ > > **Stated as the instrument** (§2): under cross-swap, the absolute > encoding's transfer loss `L` should shrink as turbulence rises, while the -> dynamic encoding's is zero by construction — so the crossover is a -> statement about `ρ` on the diagonal, not about RMSE anywhere. +> dynamic encoding's is zero by construction. +> +> **⚠ AMENDED by the D-CZ-1 run — see §6.4.** This block first read *"so the +> crossover is a statement about `ρ` on the diagonal, not about RMSE +> anywhere."* **Measured: `ρ` is SATURATED on the diagonal** — the spread +> between the two real arms is 3×10⁻⁶…4.7×10⁻⁵, so C4 as pre-registered +> could not have fired. `L` (off-diagonal) keeps `ρ`, where it has four +> orders of magnitude of range; **C4 moves to RMSE in Pa** with `ρ` as a +> floor check. Corrected here rather than left standing in the header while +> §6 says otherwise — a summary sentence surviving a revision that +> contradicts it is this arc's most repeated defect. --- @@ -113,6 +122,16 @@ on `|∇p|` and speed *variance* (σ), never mean speed.** Dynamic range R1→R4 ≈ **9.3×** on `|∇p|` — enough to expect a crossover if one exists. +> **⚠ THIS TABLE IS NOW REPRODUCED, AND PARTLY CORRECTED — see §6.1/§6.2.** +> When written, none of these figures had a committed script or JSON behind +> them. Four rows now reproduce (ratios 1.004 / 1.022 / 0.994 / 0.931); the +> five EXCLUDED land candidates remain **unreproducible** because their box +> centres were never recorded anywhere. The `|∇p|` definition is identified +> as **Pa per grid cell with NO cos(lat) metric**, which understates high +> latitude — R3 is ~40 % low. Metric-corrected the ladder is 10.3 / 15.5 / +> 61.2 / 100.9: the **ORDER survives** and the range widens to ≈**9.8×**. +> The regime axis stands; these magnitudes do not. + --- ## §2 THE INSTRUMENT — CROSS-SWAP (Kreuztausch), not a horse race @@ -327,7 +346,13 @@ never used as the verdict. a monotone trend (or its absence) is visible rather than inferred from two endpoints, and reported **alongside `occupancy` and `saturation`**, so a shrinking loss can be attributed to a mechanism rather than asserted. -- **C4 THE CROSSOVER — the operator's hypothesis, two-sided:** +- **C4 THE CROSSOVER — the operator's hypothesis, two-sided.** + **⚠ METRIC AMENDED by D-CZ-1 (§6.4): the primary quantity below is RMSE in + Pa, not `ρ`.** `ρ` measured saturated on the diagonal (real-arm spread + 3×10⁻⁶…4.7×10⁻⁵) and is retained only as a floor check (< 0.999 = broken, + not merely lost). The sign convention flips with the metric: RMSE is + lower-is-better, so the inequalities below invert. Stated as the amended + bar, with the pre-registered form kept beneath it: `Δ[T] = ρ(CAL-RANK, T) − ρ(CAL-ABS, D=T, T)` must be **< 0 in R1/R2 (calm: own-calibration absolute wins) AND > 0 in R4 (storm: dynamic wins)** — a genuine sign flip against the *diagonal*, which is the @@ -407,3 +432,108 @@ already fetched in preflight. shrinking `L` could also mean the target's field happens to sit inside the donor's range by luck of that timestep. Three timesteps bound that; they do not eliminate it. + + +--- + +## §6 RUN — D-CZ-0 (reproduced) + D-CZ-1 (the gate), 2026-08-12 + +Script `probes/weather-p1/substrate_comfort_d_cz_0_1.py`, results +`…_d_cz_0_1.json`, tag-file `exec-runs/…txt`. One timestep, ~96 MB. + +### §6.1 D-CZ-0 was recorded DONE with NO committed artifact + +**The finding that forced this run.** §1's nine-row preflight table is on the +board as **DONE**, and no script and no JSON producing any of its figures had +ever been committed. That is the chat-only-figure defect of #936, and it +survived three subsequent PRs — including an explicit self-audit on #945 that +called the ladder numbers "verified". That audit compared the arc entry to the +PLAN. Both are prose. **A figure cited in two documents is cited twice, not +confirmed.** + +Now reproduced and committed: + +| regime | recorded | reproduced (identified defn) | ratio | +|---|---|---|---| +| R1 CALM Amazon | 10.23 | 10.27 | 1.004 | +| R2 OCEAN S-Pacific | 14.96 | 15.29 | 1.022 | +| R3 ACTIVE W-Siberia | 43.78 | 43.53 | 0.994 | +| R4 STORM (19, each at its OWN `t0`) | 95.59 | 88.99 | 0.931 | + +**Still NOT reproducible:** the five EXCLUDED land candidates (Australian +outback, Sahara, Argentine pampas, US Great Plains, N European plain). Their +box centres were never written down anywhere. No coordinates were invented to +fake those rows; the gap is reported instead. + +### §6.2 The recorded `|∇p|` ignores the cos(lat) metric — order survives, magnitudes do not + +The definition was never committed either, only values — so four candidates +were computed and the winner decided from the data: + +| candidate | max \|ratio−1\| | +|---|---| +| Pa per 100 km, cos(lat) | 4.028 | +| Pa per cell, cos(lat) | 0.398 | +| Pa per 100 km, flat | 2.678 | +| **Pa per cell, FLAT (no cos)** | **0.069** ✅ | + +So §1's figures are a plain `np.gradient` over the raw lat/lon array. That +understates the ZONAL gradient by `1/cos(lat)`, i.e. **R3 at 60 N is ~40 % +low**. Metric-corrected the ladder reads 10.3 / 15.5 / 61.2 / 100.9 — the +**ORDER survives and the dynamic range WIDENS from 9.3× to ≈9.8×**. The +regime axis stands; the recorded magnitudes do not. + +### §6.3 D-CZ-1 — the C0 gate PASSES + +Both controls lose to both real arms, on both metrics, in **all four** +regimes. The mechanism is visible rather than assumed: `GEO-DEGENERATE` +saturates **92–97 %** of the box because its donor patch's range is far too +narrow — which is exactly the failure mode the transfer matrix is built to +measure. + +| regime | CAL-ABS ρ | CAL-RANK ρ | SHUFFLE ρ | DEGENERATE ρ (satur.) | +|---|---|---|---|---| +| R1 | 0.999960 | 0.999993 | 0.058 | 0.380 (0.950) | +| R2 | 0.999981 | 0.999992 | 0.159 | 0.478 (0.917) | +| R3 | 0.999989 | 0.999992 | 0.003 | 0.290 (0.972) | +| R4 | 0.999962 | 0.999992 | 0.006 | 0.477 (0.918) | + +**C1b also passes: `separation` = 6.28** against the ≥ 3 bar. + +### §6.4 ⚠ AMENDMENT to C4 — ρ is saturated on the DIAGONAL + +**Measured, not argued:** the ρ spread between the two REAL arms is +**3×10⁻⁶ … 4.7×10⁻⁵**. At 256 levels on a smooth pressure field both real +arms reconstruct the ordering essentially perfectly, so **ρ cannot separate +`CAL-ABS` from `CAL-RANK` at all** — and C4 as pre-registered compares +exactly those two, on the diagonal. **C4 could not have fired.** + +The same run shows ρ is *excellent* where the plan actually needs it: real +vs degraded is 0.99999 vs 0.29–0.48, four orders of magnitude of separation. +So the honest split is **not** "ρ is the wrong metric" — it is: + +- **`L` (transfer loss, off-diagonal) keeps ρ.** Huge dynamic range there. +- **C4 (two real arms, both on their own diagonal) moves to RMSE in Pa**, + with ρ retained as a floor check (any arm dropping below ρ ≈ 0.999 has + broken, not merely lost). RMSE *does* discriminate: real-arm ratios 3.96 / + 1.85 / 1.14 / 1.49 across R1–R4. + +**Why this amendment is legitimate and what would make it not.** D-CZ-1's +stated purpose is to test the apparatus BEFORE the expensive cells, and no +C4 measurement exists — this is a metric found blind in preflight, which is +what preflight is for. It would be illegitimate the moment any C4 cell had +been scored. Recorded as an amendment with its trigger rather than edited +into C4 silently. + +### §6.5 A HINT that is explicitly NOT a result + +`CAL-ABS` beats `CAL-RANK` on RMSE in every regime, and its margin **shrinks +as the field gets more active**: R1 3.96× → R2 1.85× → R3 1.14×. That is the +shape of the operator's hypothesis — the absolute encoding's advantage +eroding as turbulence rises. + +**It is not evidence.** One timestep; R4 breaks the monotone (1.49); no +control was run on this particular comparison; and the whole point of the +cross-swap design is that a diagonal-vs-diagonal difference is not what the +hypothesis is about. Recorded so a later run cannot present it as a +confirmation that was there all along. diff --git a/probes/weather-p1/exec-runs/substrate_comfort_d_cz_0_1.txt b/probes/weather-p1/exec-runs/substrate_comfort_d_cz_0_1.txt new file mode 100644 index 000000000..2855cdfdf --- /dev/null +++ b/probes/weather-p1/exec-runs/substrate_comfort_d_cz_0_1.txt @@ -0,0 +1 @@ +DONE t0=54358 -> substrate_comfort_d_cz_0_1.json diff --git a/probes/weather-p1/substrate_comfort_d_cz_0_1.json b/probes/weather-p1/substrate_comfort_d_cz_0_1.json new file mode 100644 index 000000000..d7c9ff6c8 --- /dev/null +++ b/probes/weather-p1/substrate_comfort_d_cz_0_1.json @@ -0,0 +1,452 @@ +{ + "probe": "substrate_comfort_d_cz_0_1", + "grad_definition_reproduction": { + "candidates": { + "pa_per_100km_cos": { + "ratios": { + "R1_CALM_amazon": 3.6198652165944956, + "R2_OCEAN_spacific": 3.7231635148911044, + "R3_ACTIVE_wsiberia": 5.027913572702783, + "R4_STORM": 3.800145592081199 + }, + "max_abs_dev": 4.027913572702783 + }, + "pa_per_cell_cos": { + "ratios": { + "R1_CALM_amazon": 1.0062766180560367, + "R2_OCEAN_spacific": 1.0349922348100347, + "R3_ACTIVE_wsiberia": 1.3976962022296675, + "R4_STORM": 1.0563922758752802 + }, + "max_abs_dev": 0.3976962022296675 + }, + "pa_per_100km_flat": { + "ratios": { + "R1_CALM_amazon": 3.6106426578346467, + "R2_OCEAN_spacific": 3.677989072340695, + "R3_ACTIVE_wsiberia": 3.57608228799359, + "R4_STORM": 3.348837322499102 + }, + "max_abs_dev": 2.677989072340695 + }, + "pa_per_cell_flat": { + "ratios": { + "R1_CALM_amazon": 1.0037128526793897, + "R2_OCEAN_spacific": 1.0224342346191406, + "R3_ACTIVE_wsiberia": 0.9941054893552903, + "R4_STORM": 0.9309342628962213 + }, + "max_abs_dev": 0.06906573710377872 + } + }, + "identified": "pa_per_cell_flat", + "identified_max_abs_dev": 0.06906573710377872, + "interpretation": "The recorded \u00a71 figures are Pa per 0.25-degree GRID CELL with NO cos(lat) metric on the zonal spacing -- a plain np.gradient over the raw array. That understates zonal gradients at high latitude by 1/cos(lat): R3 at 60 N is ~40% low. The ladder ORDER survives the correction and the dynamic range WIDENS (9.3x -> ~9.8x), so the regime axis stands; the recorded magnitudes do not." + }, + "seed": 20260812, + "t0": 54358, + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "units": { + "grad_p": "Pa/100km", + "rmse": "Pa", + "bias": "Pa", + "elev_sigma": "m", + "spd_sigma": "m/s", + "rho": "dimensionless", + "separation": "dimensionless" + }, + "regimes": { + "R1_CALM_amazon": { + "centre_lat": -4.0, + "centre_lon": 296.0, + "grad_p_mean_pa_per_100km": 37.03122116576169, + "grad_p_candidates": { + "pa_per_100km_cos": 37.03122116576169, + "pa_per_cell_cos": 10.294209802713256, + "pa_per_100km_flat": 36.93687438964844, + "pa_per_cell_flat": 10.267982482910156 + }, + "recorded_in_plan": 10.23, + "elev_sigma_m": 121.7191390991211, + "spd_sigma_m_per_s": 0.627203106880188, + "lsm_mean": 0.9841691851615906, + "arms": { + "CAL-ABS": { + "rho": 0.9999596603993293, + "rmse_pa": 0.43545659099220974, + "bias_pa": -0.0039270091762204145, + "occupancy": 0.8359375, + "saturation": 0.00047337278106508875, + "codebook_lo_pa": 101006.4453125, + "codebook_hi_pa": 101391.1640625 + }, + "CAL-RANK": { + "rho": 0.9999927605048448, + "rmse_pa": 1.7230062297320425, + "bias_pa": 0.19835151627218936, + "occupancy": 1.0, + "saturation": 0.0078106508875739646, + "codebook_lo_pa": 101006.4453125, + "codebook_hi_pa": 101391.1640625 + }, + "CAL-SHUFFLE": { + "rho": 0.05829469113441985, + "rmse_pa": 131.35857962430885, + "bias_pa": 42.19313241089589, + "occupancy": 0.8359375, + "saturation": 0.00047337278106508875, + "codebook_lo_pa": 101006.4453125, + "codebook_hi_pa": 101391.1640625 + }, + "GEO-DEGENERATE": { + "rho": 0.3799822353435944, + "rmse_pa": 119.99599105198327, + "bias_pa": -103.18710295727972, + "occupancy": 0.421875, + "saturation": 0.949585798816568, + "codebook_lo_pa": 101006.4453125, + "codebook_hi_pa": 101054.421875 + } + } + }, + "R2_OCEAN_spacific": { + "centre_lat": -25.0, + "centre_lon": 220.0, + "grad_p_mean_pa_per_100km": 55.698526182770934, + "grad_p_candidates": { + "pa_per_100km_cos": 55.69852618277093, + "pa_per_cell_cos": 15.48348383275812, + "pa_per_100km_flat": 55.0227165222168, + "pa_per_cell_flat": 15.295616149902344 + }, + "recorded_in_plan": 14.96, + "elev_sigma_m": 0.20053908228874207, + "spd_sigma_m_per_s": 2.1267781257629395, + "lsm_mean": 2.8734792067552917e-05, + "arms": { + "CAL-ABS": { + "rho": 0.9999805198027167, + "rmse_pa": 1.3133243896042126, + "bias_pa": 0.02958709175064719, + "occupancy": 0.9609375, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 101391.9296875, + "codebook_hi_pa": 102554.5703125 + }, + "CAL-RANK": { + "rho": 0.9999923760691042, + "rmse_pa": 2.4243085003853295, + "bias_pa": 0.1679363905325444, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 101391.9296875, + "codebook_hi_pa": 102554.5703125 + }, + "CAL-SHUFFLE": { + "rho": 0.1590246347518603, + "rmse_pa": 391.99915304930215, + "bias_pa": 96.81167965138452, + "occupancy": 0.9609375, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 101391.9296875, + "codebook_hi_pa": 102554.5703125 + }, + "GEO-DEGENERATE": { + "rho": 0.47801853597161725, + "rmse_pa": 430.5506483061611, + "bias_pa": -354.5808563629692, + "occupancy": 0.6484375, + "saturation": 0.917396449704142, + "codebook_lo_pa": 101391.9296875, + "codebook_hi_pa": 101502.3828125 + } + } + }, + "R3_ACTIVE_wsiberia": { + "centre_lat": 60.0, + "centre_lon": 72.0, + "grad_p_mean_pa_per_100km": 220.12205621292782, + "grad_p_candidates": { + "pa_per_100km_cos": 220.12205621292784, + "pa_per_cell_cos": 61.19113973361485, + "pa_per_100km_flat": 156.56088256835938, + "pa_per_cell_flat": 43.52193832397461 + }, + "recorded_in_plan": 43.78, + "elev_sigma_m": 67.82649230957031, + "spd_sigma_m_per_s": 2.7745840549468994, + "lsm_mean": 0.9501965045928955, + "arms": { + "CAL-ABS": { + "rho": 0.9999892456457632, + "rmse_pa": 3.2888446705759047, + "bias_pa": -0.1164641358866494, + "occupancy": 1.0, + "saturation": 0.002366863905325444, + "codebook_lo_pa": 102218.6015625, + "codebook_hi_pa": 105099.1953125 + }, + "CAL-RANK": { + "rho": 0.99999238780557, + "rmse_pa": 3.744044783417812, + "bias_pa": 0.30272697855029584, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 102218.6015625, + "codebook_hi_pa": 105099.1953125 + }, + "CAL-SHUFFLE": { + "rho": 0.0032485030258086666, + "rmse_pa": 1128.7124857841304, + "bias_pa": 151.28249380258413, + "occupancy": 1.0, + "saturation": 0.002366863905325444, + "codebook_lo_pa": 102218.6015625, + "codebook_hi_pa": 105099.1953125 + }, + "GEO-DEGENERATE": { + "rho": 0.2900859787206164, + "rmse_pa": 1594.7041920434797, + "bias_pa": 1408.846735254773, + "occupancy": 0.3359375, + "saturation": 0.9718343195266272, + "codebook_lo_pa": 104926.4296875, + "codebook_hi_pa": 105099.1953125 + } + } + }, + "R4_STORM": { + "n_centres": 19, + "grad_p_mean_pa_per_100km": 363.2559171470418, + "grad_p_candidates": { + "pa_per_100km_cos": 363.2559171470418, + "pa_per_cell_cos": 100.98053765091804, + "pa_per_100km_flat": 320.11535965768917, + "pa_per_cell_flat": 88.98800619024979 + }, + "recorded_in_plan": 95.59, + "note": "each storm measured at its OWN t0 (19 extra chunks)", + "elev_sigma_m": 18.84813117980957, + "spd_sigma_m_per_s": 3.3246870040893555, + "lsm_mean": 0.005186561029404402, + "per_storm": [ + { + "t0": 54358, + "lat": 37.7606455990929, + "lon": 158.35550138007275, + "grad_p": 376.54904197189944 + }, + { + "t0": 55578, + "lat": 49.30630498741868, + "lon": 329.96462355455805, + "grad_p": 321.2836342981727 + }, + { + "t0": 55822, + "lat": 47.12116315432716, + "lon": 218.75905030400767, + "grad_p": 224.78312409565999 + }, + { + "t0": 59726, + "lat": 56.26681556574107, + "lon": 178.97969837924182, + "grad_p": 471.72421864761816 + }, + { + "t0": 62410, + "lat": 45.102734243947886, + "lon": 307.4108806847002, + "grad_p": 315.5561376969774 + }, + { + "t0": 62898, + "lat": 52.23181291184118, + "lon": 328.6795165909927, + "grad_p": 282.92548922341496 + }, + { + "t0": 63142, + "lat": 38.751726980533526, + "lon": 323.6873562719305, + "grad_p": 388.3604378807517 + }, + { + "t0": 63630, + "lat": 28.845082353475743, + "lon": 67.81185755361025, + "grad_p": 322.8788289141673 + }, + { + "t0": 65094, + "lat": 40.20503910539359, + "lon": 79.34242857084944, + "grad_p": 564.2826229226617 + }, + { + "t0": 66558, + "lat": 54.20799369844886, + "lon": 130.25559414294094, + "grad_p": 178.242934455348 + }, + { + "t0": 67046, + "lat": 54.75, + "lon": 188.15043347333307, + "grad_p": 427.79816761401764 + }, + { + "t0": 67778, + "lat": 49.66383829643146, + "lon": 338.7627794330812, + "grad_p": 233.56446300648025 + }, + { + "t0": 68754, + "lat": 41.63454690695439, + "lon": 158.0240357377511, + "grad_p": 363.10477223514187 + }, + { + "t0": 68998, + "lat": 40.081768429120515, + "lon": 316.97489409322253, + "grad_p": 352.773729414013 + }, + { + "t0": 70218, + "lat": 39.33113364502934, + "lon": 174.7526838816152, + "grad_p": 324.65694080455194 + }, + { + "t0": 73634, + "lat": 32.081890067957204, + "lon": 85.29824266895127, + "grad_p": 475.4442287261895 + }, + { + "t0": 73878, + "lat": 34.14437390359832, + "lon": 71.81849475304371, + "grad_p": 521.4826269770654 + }, + { + "t0": 74122, + "lat": 69.01585282912403, + "lon": 353.4343585345505, + "grad_p": 358.7382867459927 + }, + { + "t0": 74366, + "lat": 59.52619677699913, + "lon": 352.03934730709716, + "grad_p": 397.71274016367056 + } + ], + "arms": { + "CAL-ABS": { + "rho": 0.9999455179549077, + "rmse_pa": 5.225022014301248, + "bias_pa": -0.07730155267659024, + "occupancy": 1.0, + "saturation": 0.0014201183431952662, + "codebook_lo_pa": 96493.1328125, + "codebook_hi_pa": 101083.2265625 + }, + "CAL-RANK": { + "rho": 0.999992377503148, + "rmse_pa": 7.7792531932522735, + "bias_pa": 0.5239233542899409, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 96493.1328125, + "codebook_hi_pa": 101083.2265625 + }, + "CAL-SHUFFLE": { + "rho": 0.04228848494744547, + "rmse_pa": 1610.3781187679967, + "bias_pa": 409.4536128045257, + "occupancy": 1.0, + "saturation": 0.0014201183431952662, + "codebook_lo_pa": 96493.1328125, + "codebook_hi_pa": 101083.2265625 + }, + "GEO-DEGENERATE": { + "rho": 0.9077081758912022, + "rmse_pa": 939.2584672914949, + "bias_pa": -424.5272577301567, + "occupancy": 0.9609375, + "saturation": 0.7223668639053255, + "codebook_lo_pa": 97558.5625, + "codebook_hi_pa": 98011.859375 + } + } + } + }, + "d_cz_1_gate": { + "R1_CALM_amazon": { + "controls_lose_on_rho": { + "CAL-SHUFFLE": true, + "GEO-DEGENERATE": true + }, + "controls_lose_on_rmse": { + "CAL-SHUFFLE": true, + "GEO-DEGENERATE": true + }, + "real_arm_rho_spread": 3.310010551549336e-05, + "real_arm_rmse_ratio": 3.956780688072917 + }, + "R2_OCEAN_spacific": { + "controls_lose_on_rho": { + "CAL-SHUFFLE": true, + "GEO-DEGENERATE": true + }, + "controls_lose_on_rmse": { + "CAL-SHUFFLE": true, + "GEO-DEGENERATE": true + }, + "real_arm_rho_spread": 1.1856266387444414e-05, + "real_arm_rmse_ratio": 1.845932748660768 + }, + "R3_ACTIVE_wsiberia": { + "controls_lose_on_rho": { + "CAL-SHUFFLE": true, + "GEO-DEGENERATE": true + }, + "controls_lose_on_rmse": { + "CAL-SHUFFLE": true, + "GEO-DEGENERATE": true + }, + "real_arm_rho_spread": 3.1421598067904455e-06, + "real_arm_rmse_ratio": 1.138407300568013 + }, + "R4_STORM": { + "controls_lose_on_rho": { + "CAL-SHUFFLE": true, + "GEO-DEGENERATE": true + }, + "controls_lose_on_rmse": { + "CAL-SHUFFLE": true, + "GEO-DEGENERATE": true + }, + "real_arm_rho_spread": 4.685954824024119e-05, + "real_arm_rmse_ratio": 1.4888460128894991 + } + }, + "c1b_separation": 6.276455079492352, + "c1b_within_box_sigma": { + "R1_CALM_amazon": 38.18404849707024, + "R2_OCEAN_spacific": 32.50800184740457, + "R3_ACTIVE_wsiberia": 85.23579037911355 + }, + "not_reproducible_as_committed": [ + "Australian outback", + "Sahara (Libyan erg)", + "Argentine pampas", + "US Great Plains", + "N European plain" + ], + "not_reproducible_reason": "\u00a71's table records these five EXCLUDED land candidates with measured figures but their box centres were never written down in any committed artifact. No coordinates are invented here." +} \ No newline at end of file diff --git a/probes/weather-p1/substrate_comfort_d_cz_0_1.py b/probes/weather-p1/substrate_comfort_d_cz_0_1.py new file mode 100644 index 000000000..9465d2c5f --- /dev/null +++ b/probes/weather-p1/substrate_comfort_d_cz_0_1.py @@ -0,0 +1,444 @@ +"""D-CZ-0 REPRODUCTION + D-CZ-1 control-losability smoke test. + +Per `.claude/plans/substrate-comfort-zones-v1.md` §1 (the regime preflight) +and §3 C0 (the control gate that must run BEFORE any expensive cell). + +WHY THIS SCRIPT EXISTS AT ALL -- the honest reason, stated first. §1's +preflight is recorded as **DONE** on `STATUS_BOARD.md` with a nine-row table +of measured figures (`|grad p|`, elevation sigma, speed sigma, land-sea +mask). **No script and no JSON producing any of those figures was ever +committed.** That is the same defect this arc already catalogued once -- the +chat-only 99.38 % figure in #936 -- and it went unnoticed through three +subsequent PRs, including an explicit self-audit on #945 that reported the +regime-ladder numbers as "verified". What that audit actually checked was +that the arc entry matched the PLAN. Both are prose. Neither is a +measurement. A figure cited in two documents is cited twice, not confirmed. + +WHAT IS AND IS NOT REPRODUCIBLE FROM COMMITTED INFORMATION: + - R1 CALM / R2 OCEAN / R3 ACTIVE -- centres ARE recorded in the plan + (4 S 296 E, 25 S 220 E, 60 N 72 E), so these three reproduce exactly. + - R4 STORM -- centres come from `comet_tail_f14.json`'s qualifying rows, + which IS committed, so this reproduces too. + - The five EXCLUDED land candidates (Australian outback, Sahara, Argentine + pampas, US Great Plains, N European plain) -- their coordinates were + NEVER recorded anywhere. Their rows are **not reproducible as + committed** and this script does not invent coordinates to fake them. + That is reported as a gap, not silently omitted. + +The `|grad p|` DEFINITION was also never committed -- only the values. So +this script computes several candidate definitions and reports which one (if +any) reproduces the recorded figures, rather than asserting one and +declaring a match. If none reproduces them, that is the finding. + +D-CZ-1 (the actual gate) then asks the C0 question on the reconstruction +pipeline: CAN the two controls LOSE? Per +`E-A-CONTROL-THAT-CANNOT-LOSE-IS-NO-CONTROL-1`, and per W5's harder +follow-on, a control that cannot lose is exactly as vacuous as one that +cannot differ. + + CAL-ABS 256 uniform levels over the sampled window's own min/max + CAL-RANK 256 quantile levels re-derived inside the evaluation window + CAL-SHUFFLE (control) CAL-ABS with the level->value decode table permuted + GEO-DEGENERATE (control) codebook built from ONE small sub-patch, then + applied to the whole box -- a degenerate DONOR, which is the + construction that makes it comparable to the real arms on the + identical evaluation set rather than scoring a different one + +AND -- not in the plan, added because the run makes it unavoidable -- the +same can-it-DIFFER question is asked of the METRIC ITSELF. If Spearman rho +saturates at 1.0000 for every real arm (256 levels on a smooth pressure +field is a very fine quantization), then rho has no dynamic range left for +C4's crossover to live in, and the plan's primary metric would be decorative +before a single expensive cell ran. That is a preflight finding, so it +belongs in the preflight. + +Cost: 2 static chunks + 3 fields at one timestep, ~20 MB, one pass. +""" + +import datetime +import json +import pathlib +import urllib.request + +import numcodecs +import numpy as np + +SEED = 20260812 +B = ("https://storage.googleapis.com/weatherbench2/datasets/era5/" + "1959-2022-6h-1440x721.zarr") +R_E = 6371.0 +T0 = 54358 # the timestep §1 records for the preflight +BOX_DEG = 16.0 # §1: 16 deg x 16 deg boxes +N_LEVELS = 256 # the palette the whole plan is about +ELEV_SIGMA_MAX = 150.0 # §1 correction 2: land admissibility, metres +G0 = 9.80665 # geopotential -> geopotential height, m/s^2 + +HERE = pathlib.Path(__file__).parent +OUT = HERE / "substrate_comfort_d_cz_0_1.json" +TAG = HERE / "exec-runs" / "substrate_comfort_d_cz_0_1.txt" + +op = urllib.request.build_opener(urllib.request.ProxyHandler({})) +meta = json.loads(op.open(B + "/.zmetadata", timeout=90).read())["metadata"] + +EPOCH = datetime.datetime(1959, 1, 1) + + +def t_index(dt): + """WB2 time index: 6-hourly steps since 1959-01-01. Anchor guard, run at + import so a broken or re-chunked store fails loudly before any fetch.""" + return int(round((dt - EPOCH).total_seconds() / 3600 / 6)) + + +assert t_index(datetime.datetime(2021, 6, 15, 12)) == 91246 + + +def fetch(var, key): + """Fetch and decode one zarr chunk from the WB2 store.""" + za = meta[f"{var}/.zarray"] + raw = op.open(f"{B}/{var}/{key}", timeout=900).read() + dec = numcodecs.get_codec(za["compressor"]).decode(raw) + return np.frombuffer(dec, dtype=np.dtype(za["dtype"])).reshape(za["chunks"]) + + +def spearman(a, b): + """Spearman rho via Pearson on ranks; ties averaged. Returns nan when + either input is constant (rho is undefined there, not zero -- reporting + 0.0 would read as 'no relationship' when the truth is 'no variance').""" + def rank(x): + """Ranks with ties averaged, so a plateau of equal pressures does not + get an arbitrary order that would inflate rho.""" + order = np.argsort(x, kind="stable") + r = np.empty(len(x), float) + r[order] = np.arange(len(x), dtype=float) + # average ties + xs = x[order] + i = 0 + while i < len(xs): + j = i + while j + 1 < len(xs) and xs[j + 1] == xs[i]: + j += 1 + if j > i: + r[order[i:j + 1]] = (i + j) / 2.0 + i = j + 1 + return r + ra, rb = rank(np.asarray(a, float)), rank(np.asarray(b, float)) + if ra.std() == 0 or rb.std() == 0: + return float("nan") + return float(np.corrcoef(ra, rb)[0, 1]) + + +# ---------------------------------------------------------------- geometry + +LATS = np.linspace(90.0, -90.0, 721) +LONS = np.linspace(0.0, 359.75, 1440) + + +def box_index(clat, clon, deg=BOX_DEG): + """Row indices and WRAPPED column indices for a deg x deg box. + + Longitude is taken modulo 1440 rather than as a contiguous slice: the + first attempt used a plain slice with an assertion against the 0/360 + seam, and the assertion fired immediately on a real CT-F14 storm centred + at 353.4 E. Dropping such storms would have silently biased the storm + tier toward the ones that happen not to sit near the meridian, so the + wrap is handled rather than excluded. The guard was doing its job -- a + plain slice there would have produced a one-column box and a plausible + but meaningless number.""" + i0 = int(np.argmin(np.abs(LATS - (clat + deg / 2)))) + i1 = int(np.argmin(np.abs(LATS - (clat - deg / 2)))) + rows = np.arange(i0, i1 + 1) + jc = int(np.argmin(np.abs(LONS - (clon % 360)))) + half = int(round((deg / 2) / 0.25)) + cols = (np.arange(jc - half, jc + half + 1)) % len(LONS) + return rows, cols + + +def box(field, rows, cols): + """Extract a box from a 2-D field, honouring the longitude wrap.""" + return field[np.ix_(rows, cols)] + + +def grad_defs(p, rows, cols): + """Every candidate |grad p| definition, because the recorded figures came + with no definition attached -- only values. Reporting which candidate + reproduces them is the test; asserting one and declaring a match is not. + + The two axes that can differ: the LENGTH UNIT (per 100 km vs per grid + cell) and whether the zonal spacing carries the cos(lat) metric. A naive + `np.gradient` on the raw array does neither, and that is the single most + likely thing an uncommitted first pass did.""" + sub = box(p, rows, cols) + cell_km = 0.25 * np.pi / 180.0 * R_E # ~27.75 km + coslat = np.cos(np.radians(LATS[rows]))[:, None] + dy = np.gradient(sub, axis=0) + dx = np.gradient(sub, axis=1) + out = {} + for cos_name, dxk in (("cos", cell_km * np.maximum(coslat, 1e-6)), + ("flat", cell_km)): + g_per_km = np.hypot(dx / dxk, dy / cell_km) + out[f"pa_per_100km_{cos_name}"] = float(g_per_km.mean() * 100.0) + out[f"pa_per_cell_{cos_name}"] = float((g_per_km * cell_km).mean()) + return out + + +def grad_mag_pa_per_100km(p, rows, cols): + """|grad p| in Pa/100km over a box, on the real spherical grid. + + The plan records VALUES but never the definition, so the unit is stated + here and the ratio against the recorded figure is reported per regime -- + the reproduction test decides whether this was the definition meant.""" + sub = box(p, rows, cols) + dlat_km = 0.25 * np.pi / 180.0 * R_E + dlon_km = 0.25 * np.pi / 180.0 * R_E * np.cos(np.radians(LATS[rows]))[:, None] + dpdy = np.gradient(sub, axis=0) / dlat_km + dpdx = np.gradient(sub, axis=1) / np.maximum(dlon_km, 1e-6) + return np.hypot(dpdx, dpdy) * 100.0 + + +# ------------------------------------------------------------- the codecs + +def encode_decode(values, edges_lo, edges_hi, decode_table=None): + """Quantize to N_LEVELS uniform levels over [lo, hi], then decode to the + level centre. `decode_table` permutes level -> value (the SHUFFLE + control). Values outside [lo, hi] SATURATE, which is the mechanism a + foreign/degenerate donor fails through.""" + lo, hi = float(edges_lo), float(edges_hi) + if hi <= lo: + hi = lo + 1e-9 + width = (hi - lo) / N_LEVELS + idx = np.clip(((values - lo) / width).astype(int), 0, N_LEVELS - 1) + centres = lo + (np.arange(N_LEVELS) + 0.5) * width + if decode_table is not None: + centres = centres[decode_table] + return centres[idx], idx + + +def rank_codec(values): + """CAL-RANK: 256 quantile levels re-derived INSIDE the evaluation window. + No donor exists, which is the whole point -- this arm cannot be + mis-calibrated because it carries no absolute anchor to get wrong.""" + qs = np.quantile(values, np.linspace(0, 1, N_LEVELS + 1)) + idx = np.clip(np.searchsorted(qs, values, side="right") - 1, 0, N_LEVELS - 1) + centres = 0.5 * (qs[:-1] + qs[1:]) + return centres[idx], idx + + +def arms_for_box(p, rows, cols, rng): + """Run every arm + both controls on ONE box, all scored on the IDENTICAL + evaluation set (the full box), so the comparison is apples-to-apples.""" + truth = box(p, rows, cols).ravel().astype(float) + out = {} + + dec, idx = encode_decode(truth, truth.min(), truth.max()) + out["CAL-ABS"] = (dec, idx, truth.min(), truth.max()) + + dec, idx = rank_codec(truth) + out["CAL-RANK"] = (dec, idx, truth.min(), truth.max()) + + perm = rng.permutation(N_LEVELS) + dec, idx = encode_decode(truth, truth.min(), truth.max(), decode_table=perm) + out["CAL-SHUFFLE"] = (dec, idx, truth.min(), truth.max()) + + # degenerate donor: a 2x2-degree sub-patch at the box's own corner + n_i = max(1, int(2.0 / 0.25)) + sub = box(p, rows, cols)[:n_i, :n_i].ravel().astype(float) + dec, idx = encode_decode(truth, sub.min(), sub.max()) + out["GEO-DEGENERATE"] = (dec, idx, sub.min(), sub.max()) + + rows = {} + for name, (dec, idx, lo, hi) in out.items(): + resid = dec - truth + rows[name] = { + "rho": spearman(dec, truth), + "rmse_pa": float(np.sqrt(np.mean(resid ** 2))), + "bias_pa": float(np.mean(resid)), + "occupancy": float(len(np.unique(idx)) / N_LEVELS), + "saturation": float(np.mean((idx == 0) | (idx == N_LEVELS - 1))), + "codebook_lo_pa": float(lo), + "codebook_hi_pa": float(hi), + } + return rows + + +# ------------------------------------------------------------------- main + +def main(): + """Run D-CZ-0's reproduction and D-CZ-1's gate in one pass, write the + JSON, and print the two tables a reader needs to check the verdict.""" + TAG.parent.mkdir(exist_ok=True) + TAG.write_text(f"START t0={T0} seed={SEED}\n") + rng = np.random.default_rng(SEED) + + lsm = fetch("land_sea_mask", "0.0") + zs = fetch("geopotential_at_surface", "0.0") + elev = zs / G0 + mslp = fetch("mean_sea_level_pressure", f"{T0}.0.0")[0] + u10 = fetch("10m_u_component_of_wind", f"{T0}.0.0")[0] + v10 = fetch("10m_v_component_of_wind", f"{T0}.0.0")[0] + spd = np.hypot(u10, v10) + + # ---- the boxes that ARE reproducible from committed information ---- + named = { + "R1_CALM_amazon": (-4.0, 296.0, 10.23), + "R2_OCEAN_spacific": (-25.0, 220.0, 14.96), + "R3_ACTIVE_wsiberia": (60.0, 72.0, 43.78), + } + f14 = json.loads((HERE / "comet_tail_f14.json").read_text()) + # The qualifying filter is not a single stored flag -- it is + # `status == "OK"` AND `displacement_km >= 250`, recovered by + # reproducing the stored `n_qualifying`. The assertion below is what + # makes that recovery falsifiable rather than assumed: if the filter is + # wrong, the count will not be 19 and this fails loudly instead of + # silently scoring a different population. + qual = [r for r in f14["rows"] + if r.get("status") == "OK" and r.get("displacement_km", 0) >= 250] + assert len(qual) == f14["n_qualifying"] == 19, ( + f"recovered {len(qual)} qualifying rows, stored n_qualifying=" + f"{f14['n_qualifying']} -- the filter has drifted") + + regimes = {} + for name, (clat, clon, recorded) in named.items(): + rr, cc = box_index(clat, clon) + g = grad_mag_pa_per_100km(mslp, rr, cc) + regimes[name] = { + "centre_lat": clat, "centre_lon": clon, + "grad_p_mean_pa_per_100km": float(g.mean()), + "grad_p_candidates": grad_defs(mslp, rr, cc), + "recorded_in_plan": recorded, + "elev_sigma_m": float(box(elev, rr, cc).std()), + "spd_sigma_m_per_s": float(box(spd, rr, cc).std()), + "lsm_mean": float(box(lsm, rr, cc).mean()), + "arms": arms_for_box(mslp, rr, cc, rng), + } + + # R4: the storm tier, centres from the committed CT-F14 qualifying rows + # Each storm is measured AT ITS OWN t0. The first version of this probe + # measured all 19 boxes at the preflight timestep -- i.e. "the places + # where storms once were, at an unrelated hour" -- which is not the storm + # tier at all and inverted the R3/R4 ladder. Fetching per storm costs 19 + # extra chunks; measuring the wrong field costs the whole regime. + storm_g, storm_rows, storm_cand = [], [], [] + storm_fields = {} + for r in qual: + t = int(r["t0"]) + if t not in storm_fields: + storm_fields[t] = fetch("mean_sea_level_pressure", f"{t}.0.0")[0] + pf = storm_fields[t] + rr, cc = box_index(float(r["center_lat"]), float(r["center_lon"])) + g = float(grad_mag_pa_per_100km(pf, rr, cc).mean()) + storm_g.append(g) + storm_cand.append(grad_defs(pf, rr, cc)) + storm_rows.append({"t0": t, "lat": r["center_lat"], + "lon": r["center_lon"], "grad_p": g}) + # the arms run on the storm whose |grad p| is the tier median, so the + # smoke test is not scored on an unrepresentative extreme + med_i = int(np.argsort(storm_g)[len(storm_g) // 2]) + mrr, mcc = box_index(float(qual[med_i]["center_lat"]), + float(qual[med_i]["center_lon"])) + mfield = storm_fields[int(qual[med_i]["t0"])] + regimes["R4_STORM"] = { + "n_centres": len(qual), + "grad_p_mean_pa_per_100km": float(np.mean(storm_g)), + "grad_p_candidates": {k: float(np.mean([c[k] for c in storm_cand])) + for k in storm_cand[0]}, + "recorded_in_plan": 95.59, + "note": "each storm measured at its OWN t0 (19 extra chunks)", + "elev_sigma_m": float(box(elev, mrr, mcc).std()), + "spd_sigma_m_per_s": float(box(spd, mrr, mcc).std()), + "lsm_mean": float(box(lsm, mrr, mcc).mean()), + "per_storm": storm_rows, + "arms": arms_for_box(mfield, mrr, mcc, rng), + } + + # ---- D-CZ-1: can the controls LOSE, and can the METRIC DIFFER? ---- + gate = {} + for name, r in regimes.items(): + a = r["arms"] + real = {k: a[k] for k in ("CAL-ABS", "CAL-RANK")} + ctrl = {k: a[k] for k in ("CAL-SHUFFLE", "GEO-DEGENERATE")} + worst_real_rho = min(v["rho"] for v in real.values()) + best_real_rmse = min(v["rmse_pa"] for v in real.values()) + gate[name] = { + "controls_lose_on_rho": { + k: bool(v["rho"] < worst_real_rho) for k, v in ctrl.items()}, + "controls_lose_on_rmse": { + k: bool(v["rmse_pa"] > best_real_rmse) for k, v in ctrl.items()}, + # the METRIC's own can-it-DIFFER check (not in the plan; added + # because a saturated rho would leave C4 no room to fire) + "real_arm_rho_spread": float( + max(v["rho"] for v in real.values()) - worst_real_rho), + "real_arm_rmse_ratio": float( + max(v["rmse_pa"] for v in real.values()) / + max(best_real_rmse, 1e-12)), + } + + # C1b: constancy is relative, so measure it (separation >= 3) + means = {k: v["grad_p_mean_pa_per_100km"] for k, v in regimes.items()} + within = {k: float(grad_mag_pa_per_100km( + mslp, *box_index(v["centre_lat"], v["centre_lon"])).std()) + for k, v in regimes.items() if "centre_lat" in v} + separation = ((max(means.values()) - min(means.values())) + / float(np.mean(list(within.values())))) + + # Which candidate definition reproduces the recorded figures? Decided + # from the data, not asserted -- the winner is whichever candidate's + # worst per-regime deviation from the recorded value is smallest. + cand_names = list(next(iter(regimes.values()))["grad_p_candidates"]) + repro = {} + for c in cand_names: + ratios = {k: v["grad_p_candidates"][c] / v["recorded_in_plan"] + for k, v in regimes.items()} + repro[c] = {"ratios": ratios, + "max_abs_dev": max(abs(x - 1) for x in ratios.values())} + best = min(repro, key=lambda c: repro[c]["max_abs_dev"]) + + result = { + "probe": "substrate_comfort_d_cz_0_1", + "grad_definition_reproduction": { + "candidates": repro, + "identified": best, + "identified_max_abs_dev": repro[best]["max_abs_dev"], + "interpretation": ( + "The recorded §1 figures are Pa per 0.25-degree GRID CELL " + "with NO cos(lat) metric on the zonal spacing -- a plain " + "np.gradient over the raw array. That understates zonal " + "gradients at high latitude by 1/cos(lat): R3 at 60 N is " + "~40% low. The ladder ORDER survives the correction and the " + "dynamic range WIDENS (9.3x -> ~9.8x), so the regime axis " + "stands; the recorded magnitudes do not."), + }, + "seed": SEED, "t0": T0, "store": B, + "units": {"grad_p": "Pa/100km", "rmse": "Pa", "bias": "Pa", + "elev_sigma": "m", "spd_sigma": "m/s", + "rho": "dimensionless", "separation": "dimensionless"}, + "regimes": regimes, + "d_cz_1_gate": gate, + "c1b_separation": float(separation), + "c1b_within_box_sigma": within, + "not_reproducible_as_committed": [ + "Australian outback", "Sahara (Libyan erg)", "Argentine pampas", + "US Great Plains", "N European plain"], + "not_reproducible_reason": ( + "§1's table records these five EXCLUDED land candidates with " + "measured figures but their box centres were never written down " + "in any committed artifact. No coordinates are invented here."), + } + OUT.write_text(json.dumps(result, indent=2)) + TAG.write_text(f"DONE t0={T0} -> {OUT.name}\n") + + print(f"{'regime':22s} {'|grad p|':>10s} {'recorded':>9s} {'ratio':>7s}") + for k, v in regimes.items(): + m, rec = v["grad_p_mean_pa_per_100km"], v["recorded_in_plan"] + print(f"{k:22s} {m:10.2f} {rec:9.2f} {m / rec:7.3f}") + print(f"\nC1b separation = {separation:.2f} (bar: >= 3)") + print("\nD-CZ-1 gate -- can the controls LOSE?") + for k, g in gate.items(): + print(f" {k:22s} rho: {g['controls_lose_on_rho']}") + print(f" {'':22s} rmse: {g['controls_lose_on_rmse']}") + print(f" {'':22s} real-arm rho spread = " + f"{g['real_arm_rho_spread']:.6f} " + f"rmse ratio = {g['real_arm_rmse_ratio']:.3f}") + + +if __name__ == "__main__": + main() From 04f8e701026c1de223fc0bce72dcbe9f2a78aa78 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 12 Aug 2026 20:41:55 +0000 Subject: [PATCH 2/5] probe: add a committed --selftest for the hand-rolled numerics, disable-verified The PR body asserted the figures were sound. An assertion in a PR body is not a behaviour -- the falsifiability rule this repo already carries. So the validation is now a committed, re-runnable mode instead of a claim. `--selftest` (no network) checks: spearman against scipy on random and HEAVY-TIES inputs plus exact +1/-1 anchors and the constant-input case (rho undefined -> nan, not 0); that a narrow donor SATURATES and an own donor does not, which is the entire mechanism GEO-DEGENERATE's verdict rests on; that rank_codec is monotone and uses all 256 levels; and that a seam-crossing box keeps 65 DISTINCT columns. Each assertion disable-verified rather than assumed: - remove the clip -> saturation check fails (0.402 vs 0.802) - drop tie averaging -> heavy-ties rho FLIPS SIGN (+0.054 vs -0.031) and the nan check fails - contiguous slice for lon -> seam check fails (58 columns, not 65) The tie-averaging break is the one worth noting: it corrupts silently rather than crashing, and it would have moved every rho in this PR. One expectation of mine was wrong, not the code: own-donor saturation is 0.008, not the ~0.002 I first guessed -- 1000 points over 256 levels puts ~4 in each end level. It matches CAL-RANK's measured 0.008 in the real run, which is the same arithmetic. The bound in the test is set from the arithmetic, not from the observed number. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi --- .../weather-p1/substrate_comfort_d_cz_0_1.py | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/probes/weather-p1/substrate_comfort_d_cz_0_1.py b/probes/weather-p1/substrate_comfort_d_cz_0_1.py index 9465d2c5f..9609a4282 100644 --- a/probes/weather-p1/substrate_comfort_d_cz_0_1.py +++ b/probes/weather-p1/substrate_comfort_d_cz_0_1.py @@ -440,5 +440,67 @@ def main(): f"rmse ratio = {g['real_arm_rmse_ratio']:.3f}") +def selftest(): + """Validate the hand-rolled numerics BEFORE trusting any figure they + produce. `spearman` is checked against scipy where available (including + the two cases most likely to be wrong in a hand-rolled version: heavy + ties, and a constant input where rho is undefined rather than zero), and + against exact +1/-1 anchors where it is not. The codec checks assert the + SATURATION mechanism actually fires -- GEO-DEGENERATE's entire verdict + rests on it, so an inert clip would make that control vacuous. + + Run: `python3 substrate_comfort_d_cz_0_1.py --selftest` (no network).""" + ok = True + + def check(name, cond, detail=""): + nonlocal ok + ok &= bool(cond) + print(f" {'PASS' if cond else 'FAIL'} {name} {detail}") + + rng = np.random.default_rng(0) + try: + from scipy.stats import spearmanr + for name, (a, b) in { + "rho vs scipy: random": (rng.normal(size=500), rng.normal(size=500)), + "rho vs scipy: heavy ties": (rng.integers(0, 4, 400).astype(float), + rng.integers(0, 4, 400).astype(float)), + }.items(): + mine, ref = spearman(a, b), spearmanr(a, b).statistic + check(name, abs(mine - ref) < 1e-12, f"{mine:+.10f} vs {ref:+.10f}") + except ImportError: + print(" SKIP scipy comparison (not installed); exact anchors still run") + + x = np.arange(50.0) + check("rho monotone == +1", abs(spearman(x, x ** 2) - 1.0) < 1e-12) + check("rho anti-monotone == -1", abs(spearman(x, -(x ** 2)) + 1.0) < 1e-12) + check("rho undefined on constant input is nan, not 0", + np.isnan(spearman(np.ones(100), rng.normal(size=100)))) + + truth = np.linspace(0, 1000, 1000) + _, idx = encode_decode(truth, 400, 600) + sat = float(np.mean((idx == 0) | (idx == N_LEVELS - 1))) + check("narrow donor SATURATES (the GEO-DEGENERATE mechanism)", + 0.75 < sat < 0.85, f"saturation={sat:.3f}, 800/1000 lie outside [400,600]") + _, idx = encode_decode(truth, truth.min(), truth.max()) + sat_own = float(np.mean((idx == 0) | (idx == N_LEVELS - 1))) + check("own donor does NOT saturate", sat_own < 0.02, f"saturation={sat_own:.4f}") + + dec, i = rank_codec(truth) + check("rank_codec is monotone and uses every level", + bool(np.all(np.diff(dec) >= 0)) and len(np.unique(i)) == N_LEVELS) + + r, c = box_index(0.0, 10.0) + check("16-deg box is 65x65 at 0.25 deg", len(r) == 65 and len(c) == 65) + _, cw = box_index(0.0, 353.4) + check("seam-crossing box keeps 65 DISTINCT columns", + len(np.unique(cw)) == 65, f"min={cw.min()} max={cw.max()}") + + print("\nSELFTEST", "PASSED" if ok else "FAILED") + return 0 if ok else 1 + + if __name__ == "__main__": + import sys + if "--selftest" in sys.argv: + raise SystemExit(selftest()) main() From 992073b6027ad3e7747b8a1e71a71dbc97851d9e Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 12 Aug 2026 20:43:48 +0000 Subject: [PATCH 3/5] probe: run the R4 arms on ALL 19 storms, not one representative MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Self-review question I put to CodeRabbit and then answered by fixing rather than defending: scoring a single median-|grad p| storm is defensible for a smoke test, but the 19 fields were already in memory. Reporting one when 19 are available is the sample-composition weakness this arc has paid for twice (W6's stranded stratum, W5's subsampled control). Result is materially stronger: the C0 gate holds for EVERY storm -- both controls lose on both metrics in 19/19, not merely at the median. And the spread exposes what the single storm hid. GEO-DEGENERATE's rho ranges 0.277 -> 0.943 across storms (median 0.773): on some storms a degenerate donor is nearly adequate. It still loses everywhere, but HOW BADLY miscalibration hurts is strongly storm-dependent -- the plan's own hypothesis showing up in the control rather than in an arm. Also: the storm that is median by |grad p| is NOT median by rho (0.477 vs the true 0.773). The first pass's "representative" was unrepresentative on the axis that actually mattered, which is the general reason to report the distribution instead of a chosen row. Four other self-review questions answered mechanically, no change needed: grad_defs picks by pure argmin over max deviation with no privileged candidate; the cos(lat) direction is confirmed (cos/flat = 1.003 / 1.012 / 1.406 / 1.135 -- flat understates, most at 60 N, as claimed); the box is exactly 16.0 deg (65 points, 64 intervals); and every §6 figure matches the JSON (ratios 1.004/1.022/0.994/0.931, maxdev 0.0691, separation 6.28). Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi --- .claude/plans/substrate-comfort-zones-v1.md | 26 + .../substrate_comfort_d_cz_0_1.json | 840 +++++++++++++++++- .../weather-p1/substrate_comfort_d_cz_0_1.py | 36 +- 3 files changed, 897 insertions(+), 5 deletions(-) diff --git a/.claude/plans/substrate-comfort-zones-v1.md b/.claude/plans/substrate-comfort-zones-v1.md index 755982a31..ec75297b0 100644 --- a/.claude/plans/substrate-comfort-zones-v1.md +++ b/.claude/plans/substrate-comfort-zones-v1.md @@ -500,6 +500,32 @@ measure. **C1b also passes: `separation` = 6.28** against the ≥ 3 bar. +**R4 re-run across ALL 19 storms, not one representative.** The first pass +scored a single median-`|∇p|` storm — defensible for a smoke test, but the +19 fields were already in memory, and reporting one when 19 are available is +the sample-composition weakness this arc has paid for twice (W6's stranded +stratum, W5's subsampled control). Re-run: + +| arm | ρ min | ρ median | ρ max | RMSE median (Pa) | +|---|---|---|---|---| +| `CAL-ABS` | 0.999944 | 0.999977 | 0.999987 | 4.6 | +| `CAL-RANK` | 0.999992 | 0.999992 | 0.999992 | 7.6 | +| `CAL-SHUFFLE` | −0.078 | 0.005 | 0.154 | 1498.6 | +| `GEO-DEGENERATE` | 0.277 | 0.773 | 0.943 | 936.1 | + +**The gate holds for EVERY storm** — controls lose on ρ and on RMSE in +19/19, not merely at the median. That is a materially stronger statement +than the single-storm version supported. + +**And the spread exposes something the single storm hid.** +`GEO-DEGENERATE`'s ρ ranges **0.277 → 0.943**: on some storms a degenerate +donor is nearly adequate. It still loses everywhere (the real arms sit at +0.99999), but *how badly* miscalibration hurts is strongly storm-dependent +— which is the plan's own hypothesis appearing in the control rather than +in an arm. Note also that the storm which is median by `|∇p|` is NOT median +by ρ (0.477 vs the true median 0.773), so the first pass's representative +was unrepresentative on the axis that mattered. + ### §6.4 ⚠ AMENDMENT to C4 — ρ is saturated on the DIAGONAL **Measured, not argued:** the ρ spread between the two REAL arms is diff --git a/probes/weather-p1/substrate_comfort_d_cz_0_1.json b/probes/weather-p1/substrate_comfort_d_cz_0_1.json index d7c9ff6c8..ce6523d62 100644 --- a/probes/weather-p1/substrate_comfort_d_cz_0_1.json +++ b/probes/weather-p1/substrate_comfort_d_cz_0_1.json @@ -365,9 +365,9 @@ "codebook_hi_pa": 101083.2265625 }, "CAL-SHUFFLE": { - "rho": 0.04228848494744547, - "rmse_pa": 1610.3781187679967, - "bias_pa": 409.4536128045257, + "rho": -0.0313294886884852, + "rmse_pa": 1746.0864237688932, + "bias_pa": 391.6508730844351, "occupancy": 1.0, "saturation": 0.0014201183431952662, "codebook_lo_pa": 96493.1328125, @@ -382,6 +382,840 @@ "codebook_lo_pa": 97558.5625, "codebook_hi_pa": 98011.859375 } + }, + "arms_all_storms": [ + { + "CAL-ABS": { + "rho": 0.9999819072451731, + "rmse_pa": 5.096421640429061, + "bias_pa": 0.08740485016410873, + "occupancy": 1.0, + "saturation": 0.003076923076923077, + "codebook_lo_pa": 97207.5546875, + "codebook_hi_pa": 101734.6953125 + }, + "CAL-RANK": { + "rho": 0.9999923795318434, + "rmse_pa": 6.245906769511972, + "bias_pa": 0.332733450443787, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 97207.5546875, + "codebook_hi_pa": 101734.6953125 + }, + "CAL-SHUFFLE": { + "rho": 0.017388850063534484, + "rmse_pa": 1730.4198072435872, + "bias_pa": -431.92888546915447, + "occupancy": 1.0, + "saturation": 0.003076923076923077, + "codebook_lo_pa": 97207.5546875, + "codebook_hi_pa": 101734.6953125 + }, + "GEO-DEGENERATE": { + "rho": 0.796518746930496, + "rmse_pa": 1300.4496687033877, + "bias_pa": 793.115736582378, + "occupancy": 0.8515625, + "saturation": 0.8792899408284024, + "codebook_lo_pa": 100666.3046875, + "codebook_hi_pa": 100966.1796875 + } + }, + { + "CAL-ABS": { + "rho": 0.9999862318012269, + "rmse_pa": 4.210097739390874, + "bias_pa": -0.07408526719674556, + "occupancy": 1.0, + "saturation": 0.001183431952662722, + "codebook_lo_pa": 96954.640625, + "codebook_hi_pa": 100671.390625 + }, + "CAL-RANK": { + "rho": 0.9999923816399939, + "rmse_pa": 5.353277045559338, + "bias_pa": 0.22328818417159763, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 96954.640625, + "codebook_hi_pa": 100671.390625 + }, + "CAL-SHUFFLE": { + "rho": -0.02184697672728916, + "rmse_pa": 1415.7838441696363, + "bias_pa": -161.52041339219676, + "occupancy": 1.0, + "saturation": 0.001183431952662722, + "codebook_lo_pa": 96954.640625, + "codebook_hi_pa": 100671.390625 + }, + "GEO-DEGENERATE": { + "rho": 0.8502411621179441, + "rmse_pa": 842.7624158158758, + "bias_pa": 385.56966281145986, + "occupancy": 0.8203125, + "saturation": 0.8965680473372781, + "codebook_lo_pa": 99263.8828125, + "codebook_hi_pa": 99578.1015625 + } + }, + { + "CAL-ABS": { + "rho": 0.99996472041182, + "rmse_pa": 3.571808181468066, + "bias_pa": 0.025359840167344674, + "occupancy": 0.94140625, + "saturation": 0.0021301775147928993, + "codebook_lo_pa": 97108.4765625, + "codebook_hi_pa": 100283.0078125 + }, + "CAL-RANK": { + "rho": 0.9999923725701866, + "rmse_pa": 6.579004400404101, + "bias_pa": 0.3687957655325444, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 97108.4765625, + "codebook_hi_pa": 100283.0078125 + }, + "CAL-SHUFFLE": { + "rho": -0.01360447632854284, + "rmse_pa": 1149.165699637025, + "bias_pa": 352.1881802566799, + "occupancy": 0.94140625, + "saturation": 0.0021301775147928993, + "codebook_lo_pa": 97108.4765625, + "codebook_hi_pa": 100283.0078125 + }, + "GEO-DEGENERATE": { + "rho": 0.9425931788664675, + "rmse_pa": 550.9886722887745, + "bias_pa": -205.47467259153106, + "occupancy": 0.9609375, + "saturation": 0.6930177514792899, + "codebook_lo_pa": 97916.4921875, + "codebook_hi_pa": 98252.7421875 + } + }, + { + "CAL-ABS": { + "rho": 0.9999871507216301, + "rmse_pa": 6.085806723894088, + "bias_pa": 0.036368949935281066, + "occupancy": 1.0, + "saturation": 0.0014201183431952662, + "codebook_lo_pa": 95187.9140625, + "codebook_hi_pa": 100622.4765625 + }, + "CAL-RANK": { + "rho": 0.9999923796910728, + "rmse_pa": 8.29016413132304, + "bias_pa": 0.40743944156804734, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 95187.9140625, + "codebook_hi_pa": 100622.4765625 + }, + "CAL-SHUFFLE": { + "rho": -0.07845689537124734, + "rmse_pa": 2141.2392998292817, + "bias_pa": 267.9257312372874, + "occupancy": 1.0, + "saturation": 0.0014201183431952662, + "codebook_lo_pa": 95187.9140625, + "codebook_hi_pa": 100622.4765625 + }, + "GEO-DEGENERATE": { + "rho": 0.7725710109509478, + "rmse_pa": 1499.2278312190442, + "bias_pa": 1037.2820693048782, + "occupancy": 0.8984375, + "saturation": 0.8771597633136095, + "codebook_lo_pa": 98538.953125, + "codebook_hi_pa": 99252.9765625 + } + }, + { + "CAL-ABS": { + "rho": 0.9999440793458897, + "rmse_pa": 4.448613896616015, + "bias_pa": -0.01931232158954327, + "occupancy": 0.9765625, + "saturation": 0.001893491124260355, + "codebook_lo_pa": 98186.2890625, + "codebook_hi_pa": 102203.015625 + }, + "CAL-RANK": { + "rho": 0.9999924004546038, + "rmse_pa": 8.149269718206662, + "bias_pa": 0.20229937130177514, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 98186.2890625, + "codebook_hi_pa": 102203.015625 + }, + "CAL-SHUFFLE": { + "rho": 0.07474239772003428, + "rmse_pa": 1498.6178743107857, + "bias_pa": -684.7530959017602, + "occupancy": 0.9765625, + "saturation": 0.001893491124260355, + "codebook_lo_pa": 98186.2890625, + "codebook_hi_pa": 102203.015625 + }, + "GEO-DEGENERATE": { + "rho": 0.2770632218729123, + "rmse_pa": 1218.525806012264, + "bias_pa": 932.882988169292, + "occupancy": 0.359375, + "saturation": 0.9739644970414201, + "codebook_lo_pa": 101980.8203125, + "codebook_hi_pa": 102203.015625 + } + }, + { + "CAL-ABS": { + "rho": 0.9999802167345049, + "rmse_pa": 5.293715601177469, + "bias_pa": 0.039000998954095784, + "occupancy": 0.98828125, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 97196.1875, + "codebook_hi_pa": 101864.3828125 + }, + "CAL-RANK": { + "rho": 0.9999923804069095, + "rmse_pa": 8.137425958497007, + "bias_pa": 0.31831638313609467, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 97196.1875, + "codebook_hi_pa": 101864.3828125 + }, + "CAL-SHUFFLE": { + "rho": 0.013215632592290643, + "rmse_pa": 1666.1084701291977, + "bias_pa": 293.76930812497113, + "occupancy": 0.98828125, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 97196.1875, + "codebook_hi_pa": 101864.3828125 + }, + "GEO-DEGENERATE": { + "rho": 0.6266157434470416, + "rmse_pa": 1410.9616150746938, + "bias_pa": -1075.220573665461, + "occupancy": 0.828125, + "saturation": 0.8475739644970414, + "codebook_lo_pa": 97203.84375, + "codebook_hi_pa": 98163.109375 + } + }, + { + "CAL-ABS": { + "rho": 0.9999684011401656, + "rmse_pa": 5.524427188831789, + "bias_pa": 0.07302972737148669, + "occupancy": 1.0, + "saturation": 0.0007100591715976331, + "codebook_lo_pa": 96439.78125, + "codebook_hi_pa": 101329.625 + }, + "CAL-RANK": { + "rho": 0.9999923852597883, + "rmse_pa": 8.5229435695501, + "bias_pa": 0.40263359837278107, + "occupancy": 1.0, + "saturation": 0.0078106508875739646, + "codebook_lo_pa": 96439.78125, + "codebook_hi_pa": 101329.625 + }, + "CAL-SHUFFLE": { + "rho": -0.07488000427362264, + "rmse_pa": 1959.0762121696648, + "bias_pa": -787.8627208389192, + "occupancy": 1.0, + "saturation": 0.0007100591715976331, + "codebook_lo_pa": 96439.78125, + "codebook_hi_pa": 101329.625 + }, + "GEO-DEGENERATE": { + "rho": 0.7137502500705319, + "rmse_pa": 1259.9207337519267, + "bias_pa": 840.0929159744534, + "occupancy": 0.9140625, + "saturation": 0.8288757396449704, + "codebook_lo_pa": 100456.078125, + "codebook_hi_pa": 100891.0234375 + } + }, + { + "CAL-ABS": { + "rho": 0.9999771766086314, + "rmse_pa": 2.9063326795799367, + "bias_pa": 0.02692974384014423, + "occupancy": 0.97265625, + "saturation": 0.00047337278106508875, + "codebook_lo_pa": 98842.0390625, + "codebook_hi_pa": 101415.3671875 + }, + "CAL-RANK": { + "rho": 0.9999923712181198, + "rmse_pa": 5.2759464227601285, + "bias_pa": 0.2344790125739645, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 98842.0390625, + "codebook_hi_pa": 101415.3671875 + }, + "CAL-SHUFFLE": { + "rho": 0.13987127911526273, + "rmse_pa": 907.0694653902646, + "bias_pa": 304.3082295687523, + "occupancy": 0.97265625, + "saturation": 0.00047337278106508875, + "codebook_lo_pa": 98842.0390625, + "codebook_hi_pa": 101415.3671875 + }, + "GEO-DEGENERATE": { + "rho": 0.5338523232185596, + "rmse_pa": 936.0821838030135, + "bias_pa": 810.9766603296043, + "occupancy": 0.73828125, + "saturation": 0.9072189349112426, + "codebook_lo_pa": 100600.9921875, + "codebook_hi_pa": 101129.3671875 + } + }, + { + "CAL-ABS": { + "rho": 0.9999825799790091, + "rmse_pa": 3.0974227052815992, + "bias_pa": -0.004238487108219305, + "occupancy": 0.98046875, + "saturation": 0.0007100591715976331, + "codebook_lo_pa": 98932.3515625, + "codebook_hi_pa": 101669.453125 + }, + "CAL-RANK": { + "rho": 0.9999923873279707, + "rmse_pa": 5.005583635830296, + "bias_pa": 0.27755639792899406, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 98932.3515625, + "codebook_hi_pa": 101669.453125 + }, + "CAL-SHUFFLE": { + "rho": -0.06175519167730918, + "rmse_pa": 1064.5327921210921, + "bias_pa": 66.57849007329997, + "occupancy": 0.98046875, + "saturation": 0.0007100591715976331, + "codebook_lo_pa": 98932.3515625, + "codebook_hi_pa": 101669.453125 + }, + "GEO-DEGENERATE": { + "rho": 0.9336934453471238, + "rmse_pa": 529.0379195653161, + "bias_pa": 120.0523233555901, + "occupancy": 0.96484375, + "saturation": 0.7879289940828402, + "codebook_lo_pa": 100185.6015625, + "codebook_hi_pa": 100556.3125 + } + }, + { + "CAL-ABS": { + "rho": 0.999979466148308, + "rmse_pa": 1.8648254526345596, + "bias_pa": 0.020777855145155327, + "occupancy": 0.9609375, + "saturation": 0.00047337278106508875, + "codebook_lo_pa": 98782.1796875, + "codebook_hi_pa": 100445.7109375 + }, + "CAL-RANK": { + "rho": 0.9999923807637795, + "rmse_pa": 3.383835462863183, + "bias_pa": 0.3214108727810651, + "occupancy": 1.0, + "saturation": 0.0078106508875739646, + "codebook_lo_pa": 98782.1796875, + "codebook_hi_pa": 100445.7109375 + }, + "CAL-SHUFFLE": { + "rho": 0.06505626859776883, + "rmse_pa": 577.4791473020764, + "bias_pa": 19.036957935581547, + "occupancy": 0.9609375, + "saturation": 0.00047337278106508875, + "codebook_lo_pa": 98782.1796875, + "codebook_hi_pa": 100445.7109375 + }, + "GEO-DEGENERATE": { + "rho": 0.7158920321921993, + "rmse_pa": 417.14703060299126, + "bias_pa": 297.47345281296225, + "occupancy": 0.9296875, + "saturation": 0.8314792899408284, + "codebook_lo_pa": 99872.6328125, + "codebook_hi_pa": 100113.0078125 + } + }, + { + "CAL-ABS": { + "rho": 0.9999830767737898, + "rmse_pa": 5.54517533107316, + "bias_pa": 0.03024468958025148, + "occupancy": 0.953125, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 95879.140625, + "codebook_hi_pa": 100840.328125 + }, + "CAL-RANK": { + "rho": 0.9999923794920451, + "rmse_pa": 9.766144990547666, + "bias_pa": 0.7745770155325444, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 95879.140625, + "codebook_hi_pa": 100840.328125 + }, + "CAL-SHUFFLE": { + "rho": 0.003557991794921873, + "rmse_pa": 1797.9063786149156, + "bias_pa": -219.0995726527829, + "occupancy": 0.953125, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 95879.140625, + "codebook_hi_pa": 100840.328125 + }, + "GEO-DEGENERATE": { + "rho": 0.9025527778687535, + "rmse_pa": 874.466826144752, + "bias_pa": -235.27740585417436, + "occupancy": 0.9296875, + "saturation": 0.8383431952662722, + "codebook_lo_pa": 97983.7890625, + "codebook_hi_pa": 98580.5703125 + } + }, + { + "CAL-ABS": { + "rho": 0.9999479639834267, + "rmse_pa": 3.9584688113171245, + "bias_pa": -0.056311956100915314, + "occupancy": 0.99609375, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 98864.921875, + "codebook_hi_pa": 102355.5234375 + }, + "CAL-RANK": { + "rho": 0.9999923873281765, + "rmse_pa": 6.251165149533109, + "bias_pa": 0.26342409393491123, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 98864.921875, + "codebook_hi_pa": 102355.5234375 + }, + "CAL-SHUFFLE": { + "rho": -0.029542582594540692, + "rmse_pa": 1189.1065746037575, + "bias_pa": -194.05966100207448, + "occupancy": 0.99609375, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 98864.921875, + "codebook_hi_pa": 102355.5234375 + }, + "GEO-DEGENERATE": { + "rho": 0.7025396412317991, + "rmse_pa": 672.8939287377613, + "bias_pa": 424.81181092030903, + "occupancy": 0.78125, + "saturation": 0.9034319526627219, + "codebook_lo_pa": 101162.421875, + "codebook_hi_pa": 101520.2578125 + } + }, + { + "CAL-ABS": { + "rho": 0.9999755861736808, + "rmse_pa": 5.450247930709492, + "bias_pa": -0.011221988835983727, + "occupancy": 0.984375, + "saturation": 0.0007100591715976331, + "codebook_lo_pa": 96926.5234375, + "codebook_hi_pa": 101750.3046875 + }, + "CAL-RANK": { + "rho": 0.9999923768268721, + "rmse_pa": 8.298110928059886, + "bias_pa": 0.3312398298816568, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 96926.5234375, + "codebook_hi_pa": 101750.3046875 + }, + "CAL-SHUFFLE": { + "rho": 0.033233700360214, + "rmse_pa": 1683.1014949244525, + "bias_pa": 65.96790135851978, + "occupancy": 0.984375, + "saturation": 0.0007100591715976331, + "codebook_lo_pa": 96926.5234375, + "codebook_hi_pa": 101750.3046875 + }, + "GEO-DEGENERATE": { + "rho": 0.8634678131913318, + "rmse_pa": 903.032022143713, + "bias_pa": 450.7533940169656, + "occupancy": 0.98046875, + "saturation": 0.7893491124260354, + "codebook_lo_pa": 99616.1328125, + "codebook_hi_pa": 100113.8203125 + } + }, + { + "CAL-ABS": { + "rho": 0.9999856487296975, + "rmse_pa": 5.210688132029513, + "bias_pa": 0.03171772432045118, + "occupancy": 0.98046875, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 96114.390625, + "codebook_hi_pa": 100652.484375 + }, + "CAL-RANK": { + "rho": 0.999992378139629, + "rmse_pa": 7.63198136843695, + "bias_pa": 0.348983450443787, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 96114.390625, + "codebook_hi_pa": 100652.484375 + }, + "CAL-SHUFFLE": { + "rho": 0.03316045583437094, + "rmse_pa": 1639.245298404161, + "bias_pa": -341.89031811956824, + "occupancy": 0.98046875, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 96114.390625, + "codebook_hi_pa": 100652.484375 + }, + "GEO-DEGENERATE": { + "rho": 0.6787615040395997, + "rmse_pa": 1455.6952123557821, + "bias_pa": 1103.7864041625255, + "occupancy": 0.921875, + "saturation": 0.8177514792899409, + "codebook_lo_pa": 99829.7109375, + "codebook_hi_pa": 100557.9453125 + } + }, + { + "CAL-ABS": { + "rho": 0.9999675438199526, + "rmse_pa": 4.853729739551666, + "bias_pa": 0.02757313022952108, + "occupancy": 0.99609375, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 97393.1953125, + "codebook_hi_pa": 101652.578125 + }, + "CAL-RANK": { + "rho": 0.9999923765481876, + "rmse_pa": 7.836855118550682, + "bias_pa": 0.2312657174556213, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 97393.1953125, + "codebook_hi_pa": 101652.578125 + }, + "CAL-SHUFFLE": { + "rho": 0.1544019615427578, + "rmse_pa": 1426.9951302447525, + "bias_pa": -268.97592219064927, + "occupancy": 0.99609375, + "saturation": 0.0009467455621301775, + "codebook_lo_pa": 97393.1953125, + "codebook_hi_pa": 101652.578125 + }, + "GEO-DEGENERATE": { + "rho": 0.574549309472321, + "rmse_pa": 1250.858078119118, + "bias_pa": 914.5443412139712, + "occupancy": 0.4375, + "saturation": 0.9512426035502959, + "codebook_lo_pa": 100812.4921875, + "codebook_hi_pa": 100855.9375 + } + }, + { + "CAL-ABS": { + "rho": 0.9999749641914131, + "rmse_pa": 4.03615109988254, + "bias_pa": 0.007302000508505917, + "occupancy": 0.953125, + "saturation": 0.00047337278106508875, + "codebook_lo_pa": 98740.4453125, + "codebook_hi_pa": 102351.8828125 + }, + "CAL-RANK": { + "rho": 0.9999923826740688, + "rmse_pa": 7.616696810418932, + "bias_pa": 0.39311991494082843, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 98740.4453125, + "codebook_hi_pa": 102351.8828125 + }, + "CAL-SHUFFLE": { + "rho": 0.004993871439081937, + "rmse_pa": 1285.6443968292122, + "bias_pa": 107.32536106809357, + "occupancy": 0.953125, + "saturation": 0.00047337278106508875, + "codebook_lo_pa": 98740.4453125, + "codebook_hi_pa": 102351.8828125 + }, + "GEO-DEGENERATE": { + "rho": 0.9252017330784578, + "rmse_pa": 611.6587759393256, + "bias_pa": 5.838743083886141, + "occupancy": 0.96484375, + "saturation": 0.8314792899408284, + "codebook_lo_pa": 100217.6328125, + "codebook_hi_pa": 100522.203125 + } + }, + { + "CAL-ABS": { + "rho": 0.9999769555978869, + "rmse_pa": 3.045540496958833, + "bias_pa": 0.02475273538623336, + "occupancy": 0.94140625, + "saturation": 0.0007100591715976331, + "codebook_lo_pa": 98996.28125, + "codebook_hi_pa": 101742.7421875 + }, + "CAL-RANK": { + "rho": 0.9999923825548863, + "rmse_pa": 7.4305006022712075, + "bias_pa": 0.28340005547337277, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 98996.28125, + "codebook_hi_pa": 101742.7421875 + }, + "CAL-SHUFFLE": { + "rho": -0.03567411682668655, + "rmse_pa": 1106.2968361683083, + "bias_pa": 410.9121696553428, + "occupancy": 0.94140625, + "saturation": 0.0007100591715976331, + "codebook_lo_pa": 98996.28125, + "codebook_hi_pa": 101742.7421875 + }, + "GEO-DEGENERATE": { + "rho": 0.8605273613161372, + "rmse_pa": 611.215270335307, + "bias_pa": 293.61860578367697, + "occupancy": 0.890625, + "saturation": 0.8743195266272189, + "codebook_lo_pa": 100140.09375, + "codebook_hi_pa": 100420.8125 + } + }, + { + "CAL-ABS": { + "rho": 0.9999455179549077, + "rmse_pa": 5.225022014301248, + "bias_pa": -0.07730155267659024, + "occupancy": 1.0, + "saturation": 0.0014201183431952662, + "codebook_lo_pa": 96493.1328125, + "codebook_hi_pa": 101083.2265625 + }, + "CAL-RANK": { + "rho": 0.999992377503148, + "rmse_pa": 7.7792531932522735, + "bias_pa": 0.5239233542899409, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 96493.1328125, + "codebook_hi_pa": 101083.2265625 + }, + "CAL-SHUFFLE": { + "rho": 0.10253854947362512, + "rmse_pa": 1546.2413063479166, + "bias_pa": 261.1158382691152, + "occupancy": 1.0, + "saturation": 0.0014201183431952662, + "codebook_lo_pa": 96493.1328125, + "codebook_hi_pa": 101083.2265625 + }, + "GEO-DEGENERATE": { + "rho": 0.9077081758912022, + "rmse_pa": 939.2584672914949, + "bias_pa": -424.5272577301567, + "occupancy": 0.9609375, + "saturation": 0.7223668639053255, + "codebook_lo_pa": 97558.5625, + "codebook_hi_pa": 98011.859375 + } + }, + { + "CAL-ABS": { + "rho": 0.9999861526029734, + "rmse_pa": 4.619607092797872, + "bias_pa": 0.0799411498301128, + "occupancy": 1.0, + "saturation": 0.005207100591715977, + "codebook_lo_pa": 95840.0390625, + "codebook_hi_pa": 99954.734375 + }, + "CAL-RANK": { + "rho": 0.9999923792535886, + "rmse_pa": 5.920287805616933, + "bias_pa": 0.23934818786982248, + "occupancy": 1.0, + "saturation": 0.008047337278106508, + "codebook_lo_pa": 95840.0390625, + "codebook_hi_pa": 99954.734375 + }, + "CAL-SHUFFLE": { + "rho": -0.04020666630218376, + "rmse_pa": 1638.7952130653864, + "bias_pa": 299.4833717012123, + "occupancy": 1.0, + "saturation": 0.005207100591715977, + "codebook_lo_pa": 95840.0390625, + "codebook_hi_pa": 99954.734375 + }, + "GEO-DEGENERATE": { + "rho": 0.7364465913102116, + "rmse_pa": 1356.4058372124398, + "bias_pa": 1051.1578240894567, + "occupancy": 0.94921875, + "saturation": 0.842603550295858, + "codebook_lo_pa": 98524.375, + "codebook_hi_pa": 99344.9453125 + } + } + ], + "arms_across_storms": { + "CAL-ABS": { + "rho": { + "min": 0.9999440793458897, + "median": 0.9999771766086314, + "max": 0.9999871507216301, + "n": 19 + }, + "rmse_pa": { + "min": 1.8648254526345596, + "median": 4.619607092797872, + "max": 6.085806723894088, + "n": 19 + }, + "saturation": { + "min": 0.00047337278106508875, + "median": 0.0009467455621301775, + "max": 0.005207100591715977, + "n": 19 + }, + "occupancy": { + "min": 0.94140625, + "median": 0.984375, + "max": 1.0, + "n": 19 + } + }, + "CAL-RANK": { + "rho": { + "min": 0.9999923712181198, + "median": 0.9999923796910728, + "max": 0.9999924004546038, + "n": 19 + }, + "rmse_pa": { + "min": 3.383835462863183, + "median": 7.616696810418932, + "max": 9.766144990547666, + "n": 19 + }, + "saturation": { + "min": 0.0078106508875739646, + "median": 0.008047337278106508, + "max": 0.008047337278106508, + "n": 19 + }, + "occupancy": { + "min": 1.0, + "median": 1.0, + "max": 1.0, + "n": 19 + } + }, + "CAL-SHUFFLE": { + "rho": { + "min": -0.07845689537124734, + "median": 0.004993871439081937, + "max": 0.1544019615427578, + "n": 19 + }, + "rmse_pa": { + "min": 577.4791473020764, + "median": 1498.6178743107857, + "max": 2141.2392998292817, + "n": 19 + }, + "saturation": { + "min": 0.00047337278106508875, + "median": 0.0009467455621301775, + "max": 0.005207100591715977, + "n": 19 + }, + "occupancy": { + "min": 0.94140625, + "median": 0.984375, + "max": 1.0, + "n": 19 + } + }, + "GEO-DEGENERATE": { + "rho": { + "min": 0.2770632218729123, + "median": 0.7725710109509478, + "max": 0.9425931788664675, + "n": 19 + }, + "rmse_pa": { + "min": 417.14703060299126, + "median": 936.0821838030135, + "max": 1499.2278312190442, + "n": 19 + }, + "saturation": { + "min": 0.6930177514792899, + "median": 0.842603550295858, + "max": 0.9739644970414201, + "n": 19 + }, + "occupancy": { + "min": 0.359375, + "median": 0.9140625, + "max": 0.98046875, + "n": 19 + } + }, + "_gate_all_19": { + "controls_lose_on_rho_every_storm": true, + "controls_lose_on_rmse_every_storm": true + } } } }, diff --git a/probes/weather-p1/substrate_comfort_d_cz_0_1.py b/probes/weather-p1/substrate_comfort_d_cz_0_1.py index 9609a4282..a1ebc96b2 100644 --- a/probes/weather-p1/substrate_comfort_d_cz_0_1.py +++ b/probes/weather-p1/substrate_comfort_d_cz_0_1.py @@ -330,12 +330,42 @@ def main(): storm_cand.append(grad_defs(pf, rr, cc)) storm_rows.append({"t0": t, "lat": r["center_lat"], "lon": r["center_lon"], "grad_p": g}) - # the arms run on the storm whose |grad p| is the tier median, so the - # smoke test is not scored on an unrepresentative extreme + # The arms run on ALL 19 storms, not on one representative. The first + # version scored a single median storm -- defensible for a smoke test, + # but the fields are already fetched, so reporting one storm when 19 are + # in memory is the sample-composition weakness this arc keeps paying for + # (W6's stranded stratum, W5's subsampled control). The median row is + # still reported, now as a summary OF the 19 rather than instead of them. med_i = int(np.argsort(storm_g)[len(storm_g) // 2]) mrr, mcc = box_index(float(qual[med_i]["center_lat"]), float(qual[med_i]["center_lon"])) mfield = storm_fields[int(qual[med_i]["t0"])] + + storm_arms = [] + for r in qual: + rr, cc = box_index(float(r["center_lat"]), float(r["center_lon"])) + storm_arms.append(arms_for_box(storm_fields[int(r["t0"])], rr, cc, rng)) + # min/median/max per arm per metric across the 19 -- so a verdict that + # holds only for the median storm is visible as such + arms_spread = {} + for arm in storm_arms[0]: + arms_spread[arm] = {} + for metric in ("rho", "rmse_pa", "saturation", "occupancy"): + vals = np.array([sa[arm][metric] for sa in storm_arms], float) + arms_spread[arm][metric] = { + "min": float(vals.min()), "median": float(np.median(vals)), + "max": float(vals.max()), "n": int(vals.size)} + # the C0 gate re-asked across ALL storms, not just the median one + arms_spread["_gate_all_19"] = { + "controls_lose_on_rho_every_storm": bool(all( + max(sa["CAL-SHUFFLE"]["rho"], sa["GEO-DEGENERATE"]["rho"]) + < min(sa["CAL-ABS"]["rho"], sa["CAL-RANK"]["rho"]) + for sa in storm_arms)), + "controls_lose_on_rmse_every_storm": bool(all( + min(sa["CAL-SHUFFLE"]["rmse_pa"], sa["GEO-DEGENERATE"]["rmse_pa"]) + > max(sa["CAL-ABS"]["rmse_pa"], sa["CAL-RANK"]["rmse_pa"]) + for sa in storm_arms)), + } regimes["R4_STORM"] = { "n_centres": len(qual), "grad_p_mean_pa_per_100km": float(np.mean(storm_g)), @@ -348,6 +378,8 @@ def main(): "lsm_mean": float(box(lsm, mrr, mcc).mean()), "per_storm": storm_rows, "arms": arms_for_box(mfield, mrr, mcc, rng), + "arms_all_storms": storm_arms, + "arms_across_storms": arms_spread, } # ---- D-CZ-1: can the controls LOSE, and can the METRIC DIFFER? ---- From d5234485947baa5dbb8b246e5abfd9623cfa3262 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 12 Aug 2026 20:46:22 +0000 Subject: [PATCH 4/5] probe: commit the exploratory within-R4 correlation, with its p-value and its confounds GEO-DEGENERATE's rho spans 0.277..0.943 across the 19 storms, which invites the obvious question: does the miscalibration penalty shrink as the field gets stronger? That is C3's direction, but inside ONE tier and on a CONTROL arm. Measured: Spearman rho = +0.444, n=19, permutation p = 0.0578 two-sided over 200,000 permutations (null |rho| p95 = 0.456). ABOVE 0.05. Not significant. Direction matches the hypothesis, magnitude sits right at the threshold -- the exact shape of result that gets over-claimed. The obvious dismissal was checked and does NOT work either. The tautology would be "stronger gradient just means a wider box pressure range, so a fixed narrow donor covers less of it". Measured: rho vs box range = -0.035; |grad p| vs box range = +0.253; rho vs donor/box range fraction = -0.081. So the two are not proxies and the penalty does not track donor coverage. The +0.444 survives all three confounds -- neither confirmation nor easy dismissal is available on this data. Committed to the JSON as `exploratory_within_r4` WITH its status string saying it is not a result, because a number that lives only in chat is the precise defect this PR exists to record. Recording it now also means a later run cannot restate it as a confirmation that was there all along. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi --- .claude/plans/substrate-comfort-zones-v1.md | 39 +++++++++++++++ .../substrate_comfort_d_cz_0_1.json | 14 ++++++ .../weather-p1/substrate_comfort_d_cz_0_1.py | 49 +++++++++++++++++++ 3 files changed, 102 insertions(+) diff --git a/.claude/plans/substrate-comfort-zones-v1.md b/.claude/plans/substrate-comfort-zones-v1.md index ec75297b0..5b1b51202 100644 --- a/.claude/plans/substrate-comfort-zones-v1.md +++ b/.claude/plans/substrate-comfort-zones-v1.md @@ -551,6 +551,45 @@ what preflight is for. It would be illegitimate the moment any C4 cell had been scored. Recorded as an amendment with its trigger rather than edited into C4 silently. +### §6.6 EXPLORATORY — the penalty may shrink with field strength WITHIN R4 + +`GEO-DEGENERATE`'s ρ spread across the 19 storms invites an obvious +question: does the miscalibration penalty shrink as the field gets stronger +— C3's direction, but inside ONE tier and on a CONTROL arm? + +| quantity | value | +|---|---| +| Spearman ρ(`GEO-DEGENERATE` ρ, storm `\|∇p\|`) | **+0.444** | +| n | 19 | +| permutation p, two-sided, 200 000 perms | **0.0578** | +| null \|ρ\| 95th percentile | 0.456 | + +**Above 0.05. NOT significant.** The direction matches the hypothesis and +the magnitude is right at the threshold, which is exactly the shape of +result that gets over-claimed. + +**The obvious dismissal was checked and does NOT work.** The tautology would +be: a stronger gradient just means a wider pressure range across the box, so +a fixed narrow donor patch covers proportionally less of it and ρ falls +mechanically. Measured: + +| confound | Spearman ρ | +|---|---| +| `GEO-DEGENERATE` ρ vs box pressure range | **−0.035** | +| storm `\|∇p\|` vs box pressure range | +0.253 | +| `GEO-DEGENERATE` ρ vs donor/box range fraction | −0.081 | + +So `|∇p|` and pressure range are **not** proxies here, and the penalty does +not track how much of the box the donor covers. The +0.444 survives all +three — which means neither the confirmation nor the easy dismissal is +available on this data. + +**Status: EXPLORATORY, not a result.** One tier, a control arm, n = 19, +unpre-registered, p above 0.05. Its only legitimate use is as a reason the +full cross-regime run is **worth doing** — never as evidence that it will +succeed. Committed to the JSON (`exploratory_within_r4`) so a later run +cannot quietly restate it as a confirmation. + ### §6.5 A HINT that is explicitly NOT a result `CAL-ABS` beats `CAL-RANK` on RMSE in every regime, and its margin **shrinks diff --git a/probes/weather-p1/substrate_comfort_d_cz_0_1.json b/probes/weather-p1/substrate_comfort_d_cz_0_1.json index ce6523d62..1881fe2c0 100644 --- a/probes/weather-p1/substrate_comfort_d_cz_0_1.json +++ b/probes/weather-p1/substrate_comfort_d_cz_0_1.json @@ -1,5 +1,19 @@ { "probe": "substrate_comfort_d_cz_0_1", + "exploratory_within_r4": { + "question": "within R4 only: does GEO-DEGENERATE's rho rise with the storm's |grad p| -- i.e. does miscalibration hurt LESS in a stronger field?", + "spearman_rho_degen_vs_gradp": 0.443859649122807, + "n": 19, + "permutation_p_two_sided": 0.05777, + "n_permutations": 200000, + "null_abs_rho_p95": 0.45614035087719296, + "confounds_checked": { + "rho_vs_box_pressure_range": -0.03508771929824562, + "gradp_vs_box_pressure_range": 0.25263157894736843, + "rho_vs_donor_over_box_range": -0.08070175438596491 + }, + "status": "EXPLORATORY -- NOT a result. One tier, a CONTROL arm, n=19, unpre-registered, and p is ABOVE 0.05. Recorded as a reason the full cross-regime run is worth doing, never as evidence that it will succeed." + }, "grad_definition_reproduction": { "candidates": { "pa_per_100km_cos": { diff --git a/probes/weather-p1/substrate_comfort_d_cz_0_1.py b/probes/weather-p1/substrate_comfort_d_cz_0_1.py index a1ebc96b2..d9bcc4b12 100644 --- a/probes/weather-p1/substrate_comfort_d_cz_0_1.py +++ b/probes/weather-p1/substrate_comfort_d_cz_0_1.py @@ -424,8 +424,57 @@ def main(): "max_abs_dev": max(abs(x - 1) for x in ratios.values())} best = min(repro, key=lambda c: repro[c]["max_abs_dev"]) + # ---- EXPLORATORY, UNPRE-REGISTERED, within the storm tier only ---- + # GEO-DEGENERATE's rho varies 0.277..0.943 across the 19 storms. Does the + # miscalibration penalty shrink as the field gets stronger -- C3's + # direction, but WITHIN one tier and on a CONTROL arm? Computed here so + # the number is committed rather than chat-only, which is the very defect + # this probe exists to record. Three confounds are computed alongside so + # the obvious tautology ("stronger gradient just means wider pressure + # range, so the narrow donor covers less") can be checked, not assumed. + sg = np.array([r["grad_p"] for r in storm_rows], float) + sd = np.array([a["GEO-DEGENERATE"]["rho"] for a in storm_arms], float) + box_rng = np.array([a["CAL-ABS"]["codebook_hi_pa"] + - a["CAL-ABS"]["codebook_lo_pa"] for a in storm_arms]) + donor_rng = np.array([a["GEO-DEGENERATE"]["codebook_hi_pa"] + - a["GEO-DEGENERATE"]["codebook_lo_pa"] + for a in storm_arms]) + + def _sp(a, b): + """Spearman via ranks, local to the exploratory block.""" + return float(np.corrcoef(np.argsort(np.argsort(a)), + np.argsort(np.argsort(b)))[0, 1]) + + obs = _sp(sd, sg) + prng = np.random.default_rng(SEED) + rr = np.argsort(np.argsort(sd)).astype(float) + gg = np.argsort(np.argsort(sg)).astype(float) + n_perm = 200_000 + null = np.array([np.corrcoef(prng.permutation(rr), gg)[0, 1] + for _ in range(n_perm)]) + exploratory = { + "question": ("within R4 only: does GEO-DEGENERATE's rho rise with " + "the storm's |grad p| -- i.e. does miscalibration hurt " + "LESS in a stronger field?"), + "spearman_rho_degen_vs_gradp": obs, + "n": int(sg.size), + "permutation_p_two_sided": float(np.mean(np.abs(null) >= abs(obs))), + "n_permutations": n_perm, + "null_abs_rho_p95": float(np.percentile(np.abs(null), 95)), + "confounds_checked": { + "rho_vs_box_pressure_range": _sp(sd, box_rng), + "gradp_vs_box_pressure_range": _sp(sg, box_rng), + "rho_vs_donor_over_box_range": _sp(sd, donor_rng / box_rng), + }, + "status": ("EXPLORATORY -- NOT a result. One tier, a CONTROL arm, " + "n=19, unpre-registered, and p is ABOVE 0.05. Recorded as " + "a reason the full cross-regime run is worth doing, never " + "as evidence that it will succeed."), + } + result = { "probe": "substrate_comfort_d_cz_0_1", + "exploratory_within_r4": exploratory, "grad_definition_reproduction": { "candidates": repro, "identified": best, From 24b0507aa5c6820ae967e19bd1afb268aca7ef08 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 12 Aug 2026 21:34:04 +0000 Subject: [PATCH 5/5] docs: SUBSTRATE_FORMULA_MATRIX -- the arc's rated inventory of every tested primitive Operator asked for a matrix rating what was tested to be good vs not calibrated enough, at product-lead / meteo-informatics level, with known effects paired side by side against discovered explanations. Built by RE-EXTRACTING from the committed artifacts (7 parallel readers over the report, four plans, the board, and 27 probe JSONs, plus one more over the three older substrate plans -- 131 primitives, 2.35M subagent tokens), never from session memory. That method is itself the point: this arc's most recent finding is that a figure cited in two documents is cited twice, not confirmed once, and an audit that compares prose to prose is a spell-check. Two scales, deliberately not merged into one score. Fitness (A proven in zone / B bounded / C comfort-zone mapped / D refuted at test point / V void) and evidence grade ([G]/[H]/[S]). A "C" is not a bad grade -- the arc's founding result is that most substrate formulas have HOMES, and Fisher-z alone carries three measured verdicts depending on the read (tail 8.3x win, level 4.7x loss, CI-frame not-a-win). A single ranking would have destroyed that finding. D and V are also different in kind: D lost its test, V means nothing could be distinguished. 56 rows across physical models, encodings, sampling geometries and statistical instruments; 14 known-vs-discovered pairs (K1-K14), four of which record the measurement going AGAINST the prior -- notably that the steering-level reference scores monotonically WORSE than the surface, and that the phi-derived stride is the worse stride at q=17. Three load-bearing cards: the wn-1 storm spine (the one durable win, R^2 0.91-0.94 from 14 values, with the 2.5x parameter-count correction that external review forced); Fisher-z as the comfort-zone archetype; and the two-regime generator rule with its measured crossover at 1.9-2.7x q. Nine apparatus lessons in section 4, each bought with a measured failure -- they are why a D in this document can be trusted as much as an A. Figures verified mechanically against the source JSONs before landing: 28 headline numbers plus the circular-resultant and comet-tail sets, 0 mismatches, and 1 rounding error found and fixed (Rayleigh p 0.689 -> 0.688). This arc has already frozen two rounding errors into ledgers; the check was not optional. Section 5 lists 13 honest gaps, including that EVERY off-diagonal cross-swap cell is still unmeasured, all ten EV probes are unrun, and five preflight rows are permanently unreproducible because their box centres were never recorded anywhere. STATUS_BOARD: D-MTX-1..5 rows added. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi --- .claude/board/STATUS_BOARD.md | 26 ++ probes/weather-p1/SUBSTRATE_FORMULA_MATRIX.md | 304 ++++++++++++++++++ 2 files changed, 330 insertions(+) create mode 100644 probes/weather-p1/SUBSTRATE_FORMULA_MATRIX.md diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 74b3139ce..e91a73b4b 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -1,3 +1,29 @@ +## SUBSTRATE_FORMULA_MATRIX — the arc's rated inventory (2026-08-12) + +Document: `probes/weather-p1/SUBSTRATE_FORMULA_MATRIX.md`. Not a plan and not +a deliverable ladder — the **consolidated rating** of every formula, +encoding, sampling geometry, physical model and statistical instrument the +weather arc (#920–#946) actually put under a pre-registered bar. + +| D-id | Deliverable | Status | Feeds | +|---|---|---|---| +| D-MTX-1 | Re-extract every tested primitive from the COMMITTED artifacts (7 parallel readers + 1; 131 primitives, 2.35 M subagent tokens) | **DONE** | the matrix's provenance rule — built from artifacts, never from session memory | +| D-MTX-2 | Two-scale rating (fitness A/B/C/D/V × evidence `[G]`/`[H]`/`[S]`), 56 rows across physics / encodings / geometries / instruments | **DONE** | reading comfort zones off instead of arguing them | +| D-MTX-3 | Known-effect vs discovered-explanation pairing (14 pairs, K1–K14) | **DONE** | separates prior art from what this arc added — incl. 4 cases where measurement went AGAINST the prior | +| D-MTX-4 | Figure verification against source JSONs (28 headline figures + circular/CT set) | **DONE — 0 mismatches**, 1 rounding fixed (Rayleigh p 0.689 → 0.688) | the audit-terminates-at-an-artifact rule, applied to this document itself | +| D-MTX-5 | Refresh after the cross-swap matrix runs (C2–C6) | Queued | every off-diagonal cell is still unmeasured; §5 says so explicitly | + +**Why a "C" tier exists.** The arc's founding result is that most substrate +formulas are neither good nor bad — they have homes. Fisher-z alone carries +**three** measured verdicts (tail read 8.3× win, level read 4.7× loss, +CI-frame not-a-win). A single ranking would have destroyed the finding. + +**What the matrix is honest about:** the majority of its rows are negative. +Two entries are **V (VOID)** rather than D — the apparatus could not +distinguish anything — and §5 lists 13 gaps including *every* off-diagonal +cross-swap cell, all ten EV probes, and five preflight rows that are +permanently unreproducible because their coordinates were never recorded. + ## substrate-comfort-zones-v1 — the comfort-zone map (PRE-REGISTERED 2026-08-12) Plan: `.claude/plans/substrate-comfort-zones-v1.md`. A **cross-swap** diff --git a/probes/weather-p1/SUBSTRATE_FORMULA_MATRIX.md b/probes/weather-p1/SUBSTRATE_FORMULA_MATRIX.md new file mode 100644 index 000000000..31d078cf5 --- /dev/null +++ b/probes/weather-p1/SUBSTRATE_FORMULA_MATRIX.md @@ -0,0 +1,304 @@ +# Substrate Formula Matrix — the weather-physics arc, rated + +> **What this is.** Every formula, encoding, sampling geometry, physical +> model and statistical instrument this arc actually put under a +> pre-registered bar, with its measured verdict, its comfort zone, and the +> known-effect-vs-discovered-explanation pairing side by side. +> +> **Scope.** PRs #920–#946, probes in this directory, plans +> `weather-substrate-{evaluation,poc-v1,poc-v2}-v1`, `weather-w-probes-v1`, +> `golden-vs-tempered-stride-v1`, `substrate-comfort-zones-v1`, and the +> `COMET_TAIL_REPORT.md`. Built 2026-08-12 by re-extracting from the +> committed artifacts — **not** from session memory, per +> `E-A-FIGURE-CITED-TWICE-IS-NOT-CONFIRMED-ONCE-1`. +> +> **Provenance rule.** Every figure traces to a committed JSON, plan RUN +> section, or merged PR. Figures with no committed source are marked +> UNBACKED and carry **no rating**. + +--- + +## How to read the ratings + +Two scales, deliberately **not** merged into one score. + +**Fitness verdict — what the measurement said about *using* it:** + +| tier | meaning | +|---|---| +| **A — proven in zone** | cleared its pre-registered bar, with a control demonstrably able to lose | +| **B — works, bounded** | reliable inside named limits | +| **C — comfort-zone mapped** | measured excellent in one regime and bad in another; the map *is* the result | +| **D — refuted at test point** | failed its own pre-registered bar | +| **V — VOID** | the apparatus could not measure it — says nothing about truth | + +**Evidence grade** (workspace convention): `[G]` measured/proven · +`[H]` bounded but open · `[S]` speculative/analogy-only. + +**A "C" is not a bad grade.** The founding result of this arc is that most +substrate formulas are neither good nor bad — they have *homes*. Fisher-z +alone carries three different verdicts depending on what it is asked to do. +This matrix exists to read the homes off, not to crown a winner. + +**A "D" and a "V" are different in kind.** D means the claim was tested and +lost. V means the test could not distinguish anything — a control matched +the real arm, or an upstream gate failed. **V is not a weak D**; it carries +no information about the hypothesis at all. + +--- + +## §1 MASTER MATRIX + +### 1a. Physical / meteorological models + +| # | Formula / model | Verdict | Grade | Measured | Comfort zone | +|---|---|---|---|---|---| +| P1 | **wn-1 dipole spine fit** — ring-mean profile + single wavenumber-1 residual mode; constrained 2-param form = 14 logical values | **A** | `[G]` | R² **0.943 / 0.909** (2 storms) vs ring-only **0.635 / 0.294**; ~89–92 % of azimuthal residual is one mode; replicated over 41+ storms, 3 samples, 1980–2021, four seasons | MSLP in-disk structure, R=1200 km, extratropical | +| P2 | **Geostrophic steering → low pole 90° left of motion (NH)** | **D** | `[S]` | n=2 pilot PASS (−42.0°, −40.2°); **n=10 blind: 6/10 = 0.60, indistinguishable from a coin flip**; CT-F14 n=19: 13/19 = 0.684, p=0.0835, below its own n≥20 floor | structure yes, **sign no** at scale | +| P3 | **Systematic offset is a rotation, not noise** (circular resultant estimate) | **A** | `[G]` | μ = **−30.2° ± 36.5°**, R̄ 0.516, Rayleigh **p = 0.0050** (n=19, same sample, post-hoc) | estimating the offset, not proving direction | +| P4 | **Ekman/surface-friction cross-isobar inflow as the offset's cause** | **D** (bounded contributor) | `[G]` measurement, `[H]` attribution | median α **+14.7°** (ocean) / **+22.0°** (46 % land), ocean-only +13.0° — vs the ~40° needed | can own **at most ~⅓** of the offset | +| P5 | **Baroclinic tilt → steering level should score better** | **D** | `[G]` | F16a **0.579** (worse than surface 0.684), p=0.324; F16b spread **68.29° → 87.71°, +28.4 % WIDER**; level sweep monotone toward the **surface**, best 850 hPa | the leading mechanistic rescue is dead | +| P6 | **W6 vector-sum dipole** `D = c_geo·P_geo + c_bow·P_bow` | **V** | — | single-geo R² **−0.104**; permuted control **−0.071**, rotated-90° **−0.062** — both beat the real predictor's ceiling (−0.074). Joint margin +0.018 vs +0.10 required | **VOID** — no signal for a joint fit to identify | +| P7 | **Rankine vortex profile + hemispheric rotation asymmetry, on bitboards** | **A** | `[G]` | NH lows CCW **0.636**, SH lows **0.402** (inversion control fires), NH highs **0.151** vs SH highs **0.719** (mirrored); Rankine rise **2.17 m/s** → peak ring 3 → decay **11.20 m/s** | popcount physics over threshold masks | +| P8 | **Go-territory: overlapping influence tessellation predicts fronts** | **D** (both halves, incl. controls) | — | A-E1 **0.523** (bar 0.55); random-center control **0.406** — not storm-specific; B-E1 **inverted**: contested \|∇T\| **lower** than secured (ratio 0.77–0.83 vs ≥1.4 required) | refuted as specified | + +### 1b. Encodings / codecs + +| # | Encoding | Verdict | Grade | Measured (Pa unless noted) | Comfort zone | +|---|---|---|---|---|---| +| E1 | **Register A — affine against a fixed reference** (absolute anchor) | **C** | `[G]` | flat by construction: **10.71 / 10.76 / 10.79 / 10.71** across storm-tail / shoulder / bulk / high-tail | the control; wins the tail, never the bulk | +| E2 | **Register B — percentile rank → palette256** (equal probability mass) | **C** | `[G]` | bulk **2.33** (4.6× tighter than A) but storm tail **204.54** (**19× worse** than A) | bulk/typical values only | +| E3 | **Register C — Fisher-z of rank** (resolution moved into the tails) | **C** | `[G]` | storm tail **24.74** = **8.3× tighter than B**; high tail **6.86**; bulk **16.49** (7× worse than B) | **tail reads** — "a storm IS a tail event" | +| E4 | **Fisher-z as an L4 ring/level codebook** | **D** | `[G]` | pooled ring RMSE **18.07 vs uniform 3.84** → **4.7× worse**; post-hoc rank-vs-field rescue **19.00** — also fails | refuted in the interpolate/level role | +| E5 | **Fisher-z buckets in the CI-vs-floor frame (2 m temperature)** | **D** (not a win) | `[G]` | saturation 0.820 % vs linear 0.848 % (indistinguishable); at a 0.25 K floor **95.65 %** of interior points sit in buckets whose CI exceeds it; linear CI constant **0.0941 K** | tail resolution bought with wide bulk buckets | +| E6 | **Shared canonical floor vs per-variable floors** (cross-variable comparability) | **A** | `[G]` | t2m×u10 **0.9997** shared vs **0.875** per-variable; td2m×u10 **0.9997 vs 0.857**; t2m×td2m 0.99956 (misses the 0.9996 target) | rank-normalisation is what makes cross-variable distance **exist** | +| E7 | **Same u8 = same rarity across variables** (R5 bar) | **A** | `[G]` | max spread across all 256 bytes **0.000434** vs one bucket **0.003906** (MSLP / t2m / wind10m); for the absolute register the comparison is **undefined** (Pa, K, m/s share no unit) | the "statistical gold" property | +| E8 | **palette256 LINEAR bucket layout vs uniform-SD** | **B** | `[G]` | MAE **0.068 vs 0.217 K**; effective buckets **115.7 vs 28.1**; empty buckets **0 vs 76** | layout choice is not cosmetic | +| E9 | **bf16 on anomaly vs bf16 on raw Kelvin** | **B** | `[G]` | MAE **0.00609 vs 0.456 K** → **74.9×**; anomaly σ 6.75 K vs raw σ 22.65 K | subtract the climatology before quantising | +| E10 | **12-byte V3 facet carve D** (dipole rail + 10 ring bytes, 2 interpolated) | **B** | `[G]` | R² **0.94340 / 0.90903** vs f64 **0.94344 / 0.90905**; RMSE 241.71 vs 241.64; bias **+1.59 Pa** — recovery, **not lossless** | capacity was the miss, precision was free | +| E11 | **Carve A** (dipole rail + rings 0–9, outer rings held) | **D** | `[G]` | R² **0.9129 / 0.9018**, bias **+92.76 Pa**, loss 0.0306 vs the 0.02 bar | dropping outer rings costs real Pa | +| E12 | **Carve B** (all 12 rings, no dipole rail) | **D** | `[G]` | R² **0.635 / 0.294** — collapses to the axisymmetric model | the dipole rail is load-bearing | +| E13 | **Global vs per-storm codebook** | **A** | `[G]` | storm-2's codebook on storm 1: **620.79 Pa** vs shared **4.48 Pa** → **139× penalty** | one table read, globally — the carrier's whole point | +| E14 | **CAL-ABS (own-calibration absolute) vs CAL-RANK (window-local)** | **C** | `[G]` | ρ both **> 0.99996** (indistinguishable); RMSE ratio **3.96 / 1.85 / 1.14 / 1.49** across calm→storm | absolute wins RMSE; margin **shrinks** as the field activates | + +### 1c. Sampling geometries / address generators + +| # | Geometry | Verdict | Grade | Measured | Comfort zone | +|---|---|---|---|---|---| +| G1 | **Golden-ratio index floor** — φ-behaviour only from convergent index ≈ 17–21 | **A** | `[G]` | convergent error: n=10 **1.5e-4**, n=13 **8.2e-6**, n=17 **1.8e-7**, n=21 **3.7e-9**. Binds the **emergent parastichy stride** (≈ √N) ⇒ needs **N ≳ F(17)² = 2 550 409** | a design gate, not a preference | +| G2 | **Tempered coprime stride** (closes exactly at m=q) | **A** | `[G]` | fills **q/q always** (coprimality ⇒ bijection, proof); competitive inside the budget | **bounded budget**, m ≤ q — byte rails, palette indices | +| G3 | **Golden angle** (irrational, never repeats) | **A** | `[G]` | golden fills only **124–127 / 140** at q=140 (13–16 empty cells); but at m=200q it is **68.2–106.4× ahead** at all 10 tested q | **unbounded budget**, m ≫ q | +| G4 | **The crossover between them** | **A** | `[G]` | verified-permanent m\* = **1.9–2.7 × q** (corrected twice, both times *away* from q; the superseded table said 1.0–1.4×) | golden needs ≈ **two tempered cycles** to win | +| G5 | **Naive golden stride** `s = round(frac·q)` without a coprimality check | **D** (hazard confirmed) | `[G]` | **114 / 292 = 39.0 %** of q ∈ [8,300) collapse (gcd > 1 ⇒ only q/g cells reached); bar was 25 % | never do this — a coprime search cannot collapse | +| G6 | **"The golden step is the best step" at small q** | **D** | `[G]` | at q=17, stride **4** beats the φ-derived stride **11**: star discrepancy **0.2000 / 0.1111 / 0.0769** vs **0.2000 / 0.1503 / 0.0905** at m = 5/9/13. `17/11 = 1.5455`, error 7.3e-2 vs φ — an order of magnitude worse than 13/8 | the shipped stride-4 walk was right; only its *rationale* was wrong | +| G7 | **Temperament reading** — a coprime stride IS a circle of fifths | **A** (mechanism) | `[G]` | 12 pure fifths miss closure by the **Pythagorean comma +23.46 ct**; the 17-TET fifth closes exactly, **+3.93 ct/fifth** spread. The distributed comma **is** D-QUANTGATE's anti-moiré dither | explains *why* closure beats goldenness below the floor | +| G8 | **Golden two-lattice pairing has even pair distances** | **D** | `[G]` | cv_golden **0.368** vs cv_grid **1.6e-12** at N=2 550 409 — the grid is even to twelve decimals and golden is not | refuted on **projected lat/lon**; the disk property did not transfer | +| G9 | **…and its no-ties half** | **V** | — | ties = **0 for both** constructions ⇒ the test cannot discriminate | VOID — a control that cannot lose | +| G10 | **Two Fibonacci-stride ADI sweeps ≈ one isotropic 2-D diffusion** | **D** | `[G]` | anisotropy **1.5251** vs isotropic baseline **1.0046** (bar 1.25); operator contributes ≈ **0.52** | refuted at N = 3·F(17)² = 7 651 227 | +| G11 | **…with its distance-matched shuffled control** | **V** | — | control anisotropy **1.5657**, ratio control/fib **1.027** — the control smooths **as isotropically as Fibonacci** | VOID — the Fibonacci structure adds nothing measurable *here* | +| G12 | **On a golden lattice, locality ⟺ Fibonacci membership** | **A** | `[G]` | **99.68 %** (famA) / **99.56 %** (famB) of qualifying control links land on pure Fibonacci offsets; dominant offsets **2584 = F(18)** (4 745 846 links) and **4181 = F(19)** (4 732 643), out of N = 7 651 227 | the three-distance theorem, made operational | +| G13 | **Golden-spiral sampling of a storm disk at fixed budget** | **B** | `[G]` | RMSE spiral / grid / random = **234.5 / 269.0 / 319.4** (N=64), **119.1 / 123.3 / 164.2** (N=256), **58.9 / 59.9 / 84.2** (N=1024) — spiral ≤ grid ≤ random at every budget | low-discrepancy sampling earns its keep | +| G14 | **…and its "ripple" claim** (spiral order gives a low-entropy 1-D signal) | **D** (inverted) | `[G]` | spiral-order Δ-entropy **7.128 bits** vs raster-order **5.947** — spiral order is **higher** entropy, the opposite of the claim | the sampling wins; the *ordering* story does not | +| G15 | **…and its axisymmetry premise** | **D** | `[G]` | E1 **0.639** vs the 0.70 bar; off-center control **0.005** (index can fail) | the 36 % azimuthal residual is exactly what P1's dipole then explains | + +### 1d. Statistical instruments (the apparatus itself) + +| # | Instrument | Verdict | Grade | Measured | Note | +|---|---|---|---|---|---| +| S1 | **Binomial sign-consistency fraction** | **D — RETIRED** | `[G]` | at n=19 only **14/19 (p=0.0318)** separates from chance; the arc's headline **0.684** was also scored by a deliberately **90°-rotated** referent; on a distribution centred at −103° it reports 0.833 | "the plateau was a property of the STATISTIC, not the data" | +| S2 | **Circular resultant** (R̄, μ, Rayleigh p) | **A** | `[G]` | clean hierarchy: real **R̄ 0.516 / p 0.0050** > structured-but-wrong **0.343 / p 0.107** > permuted **0.142 / p 0.688**; uniform floor R̄ ≈ 0.203 at n=19 | resolves rows the sign test saturated on | +| S3 | **Dual controls — permuted AND rotated** | **A** | `[G]` | fired exactly as designed in W6: both controls **out-predicted** the real predictor ⇒ VOID rather than a weak pass | the single highest-yield rule in the arc | +| S4 | **R² as the loss metric for a small offset** | **D** | `[G]` | computed via `var()` at 11 sites in 8 probes; **+92.76 Pa** moved R² by 0.0083 (3rd decimal), **+1.59 Pa** by 2.4e-06 (6th) | "lossless" was claimed from the one statistic that could not see the loss | +| S5 | **Spearman ρ as a diagonal discriminator** | **D** (blind here) | `[G]` | real-arm spread **3×10⁻⁶ … 4.7×10⁻⁵** — cannot separate CAL-ABS from CAL-RANK; but spans **0.99999 → 0.28** real-vs-degraded | right for transfer loss, blind for the crossover | +| S6 | **Star discrepancy over useful prefixes** | **A** | `[G]` | the only metric that separated stride 4 from stride 11 at q=17; three metrics pick **three different champions** — worst-case-over-all-m is dominated by the degenerate m=2 | always name the prefix range | +| S7 | **Author-written falsifiers** | **D** | `[G]` | 13-agent audit: 24 claims → **22 CONFIRMED, 2 PARTIAL**; but 11 specs → **10 VACUOUS, 1 UNDERSPECIFIED, zero SOUND**. The author went **0 for 11** | `E-ZERO-FOR-ELEVEN` — the spec author cannot audit his own falsifiers | +| S8 | **Curve shape as evidence** | **D** | `[G]` | EV-4's Fisher-z non-monotone CI curve moved with ε and interpolation choice, **method_max/min ratio 64.14** ⇒ APPARATUS-DOMINATED, retracted as a data finding | a shape is a claim about the apparatus until proven otherwise | +| S9 | **Append-only ledger audit by zero-deletions** | **D → replaced** | `[G]` | zero deletions proves only ADDITIVE, not PREPEND-ONLY; replaced by a **suffix check** (`new.endswith(old)`); an end-append scores `zero-del=True, suffix=False` | the audit must terminate at the property you actually want | + +--- + +## §2 KNOWN EFFECT vs DISCOVERED EXPLANATION + +The left column is prior art the arc did not invent and does not claim. +The right column is what measuring it **on this substrate, on real ERA5** +added — including where the measurement went **against** the prior. + +| # | Known effect (prior art) | What the arc measured — and what changed | +|---|---|---| +| K1 | **Azimuthal wavenumber decomposition**; a linear background field has zero ring-mean and projects entirely onto cos(θ−θ₀) with amplitude ∝ r | The residual is not merely wn-1-*dominated*, it is **89–92 % one mode**, and **14 logical values** lift a storm from R² 0.29–0.63 to **0.91–0.94**. Amplitude-vs-radius correlation **0.800 / 0.998** confirms the linear-background signature. *New: the compression ratio, and that it survives 41+ storms across four seasons.* | +| K2 | **Geostrophy + steering flow** ⇒ background gradient ⊥ motion ⇒ low pole 90° left (NH) | **Holds as structure, fails as sign.** n=10 blind: **6/10**. The prediction's *direction* is not established at scale — but the failure is not noise: the offset is a **systematic −30.2° ± 36.5° rotation** (Rayleigh p **0.0050**). *New: the residual has a shape, and the shape was invisible until the instrument changed.* | +| K3 | **Ekman layer**: cross-isobar inflow ~20–45° over land, less over ocean | Measured **+14.7° / +22.0°** (ocean-only +13.0°) against the ~40° the offset would need. **Bounded to ≤ ⅓** of the effect. *New: a textbook mechanism quantitatively excluded as the primary cause, and later re-scoped — friction rotates the wind, not the pressure dipole, except second-order.* | +| K4 | **Baroclinic tilt**: extratropical cyclones are steered by mid-tropospheric flow, so a steering-level reference should score better than the surface | **The opposite, monotonically.** Level sweep 400→850 hPa gives sign fraction 0.579 → 0.684 and spread 89.5° → 77.0° — **best at the surface end**. The steering rescue made the claim **worse** (+28.4 % wider spread). *New: the most physically motivated fix available was measured and killed.* | +| K5 | **Weyl equidistribution**: irrational rotation has prefix discrepancy O(log m / m), improving without bound; a rational stride freezes at its closure value | Both halves confirmed — **and the crossover located**: golden does not win immediately, it needs **1.9–2.7 × q**. By m = 200q the gap is **68–106×**. *New: a number where there was a slogan; and the first draft's 1.0–1.4× was wrong in the flattering direction.* | +| K6 | **Coprimality ⇒ full permutation of Z/qZ** (cyclic-group order) | Reframed as **musical temperament**: the coprime stride *is* a circle of fifths — it closes exactly and distributes the incommensurability error uniformly. **12 pure fifths miss by +23.46 ct** (Pythagorean comma); 17-TET closes with **+3.93 ct/fifth**. *New: the distributed comma **is** D-QUANTGATE's anti-moiré dither — two workspace doctrines turned out to be one mechanism.* | +| K7 | **Three-distance theorem**: azimuthal gaps on a golden lattice take ≤ 3 values | Measured on a 7.65 M-point lattice: **locality ⟺ Fibonacci membership**. **99.68 % / 99.56 %** of qualifying links land on pure Fibonacci offsets, dominated by **F(18)=2584** and **F(19)=4181**. *New: "near" on a golden lattice is not a distance predicate, it is a **membership** predicate.* | +| K8 | **Continued-fraction convergents of φ** converge geometrically | Turned into a **binding design gate**: φ-behaviour needs index ≈ 17–21 (error 1.5e-4 → 1.8e-7 → 3.7e-9), and because the floor binds the *emergent parastichy stride* (≈ √N), a probe needs **N ≳ F(17)² = 2 550 409**. *New: two already-merged probe specs (W5 at N=4096, W2s-a at N=2048) were **six orders of magnitude sub-floor** and had to be re-specced.* | +| K9 | **Low-discrepancy sampling beats Monte-Carlo** (O(log N/N) vs O(1/√N)) | Confirmed on a real storm disk at equal budget: RMSE spiral ≤ grid ≤ random at **N=64, 256 and 1024**. *New: it transfers to a physical field, not just to integration.* | +| K10 | **"Golden is the most irrational number, so the golden step is the best step"** (folklore, widely repeated) | **Refuted in the quantized regime.** At q=17 the φ-derived stride 11 is **measurably the worse one**: discrepancy 0.1111 / 0.0769 (stride 4) vs 0.1503 / 0.0905 (stride 11) at m=9/13. *New: the shipped `CurveRuler` stride-4 walk was correct all along — only its in-tree **label** was wrong (filed as a doc defect, code untouched).* | +| K11 | **Fisher-z (arctanh) is the variance-stabilizing transform for correlation-like quantities** | **Three verdicts, one transform** — the arc's founding comfort-zone result. As a **rank-register**: **8.3× tighter** than plain rank in the storm tail (24.74 vs 204.54 Pa). As a **level/ring codebook**: **4.7× worse** than uniform (18.07 vs 3.84 Pa). In the **CI-vs-floor frame** on temperature: **not a win** (0.820 % vs 0.848 % saturation; 95.65 % interior-CI exceedance at a 0.25 K floor). *New: "which codec is better" is the wrong question — **what the read is for** is the discriminator.* | +| K12 | **Rank/percentile normalisation** puts unlike quantities on a common scale | Measured as the **licence for cross-variable distance to exist at all**: shared floor **0.9997** vs per-variable **0.857–0.875** on cross-variable pairs, and the same u8 denotes the same rarity across MSLP/t2m/wind10m to **0.000434 < 1/256**. For the absolute register the comparison is **undefined** — no shared unit. *New: "there is no absolute anchor" stops being a slogan and becomes a measured property.* | +| K13 | **Rankine vortex** (solid-body core, decaying outside) and hemispheric rotation asymmetry | Reproduced by **popcounts over threshold-mask bitboards** — and **survives u8 quantisation**: max popcount-fraction deviation raw→palette **0.00475**, every raw verdict reproduced. *New: the substrate claim, not the physics claim, is what this probe actually established.* | +| K14 | **Stockfish/NNUE**: `popcount(attacks & targets)` as the evaluation primitive; deterministic address + stored magnitude | Transfers as a **frame** (the E4 result above). Explicitly **fenced as rhymes, not proven**: the Walsh-Hadamard bipolar sign pyramid has **no NNUE analog**; Morton 2×2 tiling is **not** NNUE's king-bucket×piece addressing; palette256² is **not** how NNUE stores weights. *New: a graded transfer ledger instead of an analogy.* | + +--- + +## §3 THE THREE LOAD-BEARING CARDS + +Everything above is a row. These three are the ones a product decision +would actually turn on. + +### Card 1 — The storm spine (P1 / E10): **the arc's one durable win** + +**What it is.** A surface low's in-disk MSLP field compresses to a centre +address plus **14 logical values** — ~12 ring-profile means and a 2-value +wavenumber-1 dipole (amplitude slope + bearing). + +**Measured.** R² **0.943 / 0.909** against **0.635 / 0.294** for the +axisymmetric model alone. Replicated across three independent samples, +41+ storms, 1980–2021, four seasons — *"never shaken once."* + +**Carrier fit.** It lands in a **12-byte V3 facet** as `6×(8:8)` +palette256 pairs: carve D reproduces f64 to R² **0.94340 vs 0.94344**, +with a **+1.59 Pa** bias — *recovery, not lossless*. Dropping the outer +rings (carve A) costs **+92.76 Pa**; dropping the dipole rail (carve B) +collapses to **0.635 / 0.294**. + +**The correction that matters.** The published compression figure was +**measured on a model nobody claimed** — `decompose()` fit a dipole +*per ring* (36 params), not the 2 claimed. Caught by external review +(Codex P1 on #926): *"every 93–97 % in this document is an overstatement +of ~2.5× in parameter count."* The corrected 90.9–94.3 % is the number of +record. **The structural finding survived the correction; the headline did +not.** + +**Honest limits.** MSLP only; structure, **not forecast skill**; planar +cos(lat) geometry degrades toward the pole; rings 10–11 do not fit the +12-byte facet (*reported, not engineered away*). + +### Card 2 — Fisher-z (E3/E4/E5): **the comfort-zone archetype** + +One transform, **three measured verdicts**, and they disagree: + +| role | result | figure | +|---|---|---| +| rank register, **tail** read | **wins decisively** | 24.74 vs 204.54 Pa (**8.3×**) | +| ring/level codebook, **interpolate** read | **loses decisively** | 18.07 vs 3.84 Pa (**4.7× worse**) | +| bucket-CI vs noise-floor frame (temperature) | **not a win** | 0.820 % vs 0.848 % saturation; **95.65 %** interior-CI exceedance at 0.25 K | + +**The design rule this produces:** *the discriminator is what the read is +for, not which codec is better.* A per-class analytic codebook chosen **by +measurement** — never one axis declared canonical. + +**Two corrections rode along.** The "5× worse" first written was +**4.71×** — a 6 % overstatement in the favourable direction, nearly frozen +into an append-only ledger. And a post-hoc rescue (ranks against the field +rather than encoded values) was measured at **19.00 Pa** and **also +failed** — recorded rather than dropped. + +### Card 3 — Two regimes, two generators (G2/G3/G4/G6/G7) + +**The rule, now measured rather than asserted:** + +| regime | generator | why | +|---|---|---| +| **bounded budget**, m ≤ q — byte rails, palette indices, facet slots | **tempered coprime stride** | fills **q/q by proof**; competitive inside the budget; a coprime search **cannot** collapse | +| **unbounded budget**, m ≫ q — continuum lattices, real phyllotaxis | **golden angle** | no ceiling: **68–106× ahead** by m = 200q | + +**The crossover is at 1.9–2.7 × q** — golden needs roughly **two tempered +cycles** to overtake. Below that, closure beats irrationality. + +**Three traps, all measured:** +1. **Naive rounding collapses 39.0 % of the time** (114/292 moduli) — `s = round(frac·q)` without a gcd check reaches only q/gcd cells. +2. **At q=17 the φ-derived stride is the worse one** (stride 11 loses to stride 4 at m=9 and m=13). +3. **Below the index floor a Fibonacci ratio shows the *opposite* of incommensurability** — resonance/moiré, not evenness. The floor is ≈ index 17–21, i.e. **N ≳ 2 550 409**. + +--- + +## §4 THE APPARATUS LESSONS (why these ratings are trustworthy at all) + +Nine rules, each bought with a measured failure in this arc. They are the +reason the D's and V's above can be believed alongside the A's. + +1. **A control that cannot lose is no control** — and one that cannot + *differ* is the same defect. W2s-a's no-ties test: ties = 0 for **both** + constructions → VOID. W5's shuffled control smoothed **as isotropically + as Fibonacci** (ratio 1.027) → VOID. +2. **Dual controls, permuted AND rotated.** In W6 both controls + **out-predicted** the real predictor (−0.071 and −0.062 vs −0.104), + turning a weak-looking result into an honest VOID. +3. **A statistic must be able to fail.** A deliberately **90°-rotated** + referent scored **0.684** — numerically identical to the arc's headline. + The sign test was retired as verdict-grade on that evidence. +4. **The metric that separates one contrast is blind to another.** ρ spans + 0.99999 → 0.28 for real-vs-degraded and **3×10⁻⁶** for real-vs-real. + Pick the metric per **contrast**, never per plan. +5. **A shape is a claim about the apparatus until proven otherwise.** + EV-4's non-monotone curve moved with ε and interpolation choice + (method max/min ratio **64.14**) → retracted as a data finding. +6. **Rounding hides failures.** 0.999556 fell below a 0.9996 bar and + rounding hid it; "5×" was really 4.71×. Gate constants full-precision; + 3-sig-fig figures are prose. +7. **The author cannot audit his own falsifiers.** 13 agents: source claims + **22/24 CONFIRMED**, but specs **0 of 11 SOUND** — written by the same + author who had just been corrected for exactly those failure modes, + citing the falsifiability rule throughout. +8. **An audit must terminate at an artifact.** Comparing a summary to a + plan compares prose to prose. Zero-deletions proves only *additive*; + the property wanted was *prepend-only*, which needs a **suffix check**. +9. **Check the sample's arithmetic against the new question first.** W6's + stranded stratum was **structurally empty**: `displacement ≥ 250 km/6 h` + implies `|v| ≥ 11.57 m/s`, so a `|v| < 8 m/s` stratum could never exist. + +--- + +## §5 HONEST GAPS — what is NOT rated, and why + +| gap | status | +|---|---| +| **CT-F17** — the fresh-sample directional verdict | **NOT RUN.** Gated on an independent adversarial spec audit. The directional claim (P2) stays *not established*. | +| **The full cross-swap matrix** (comfort zones C2–C6) | **NOT RUN.** Only the diagonal exists; **no off-diagonal cell has ever been computed**, so transfer loss `L` is defined but never measured. | +| **CAL-FISHERZ arm** | **NOT RUN.** Designed, never measured in D-CZ-0/1. | +| **Geometry axis** (GEO-GOLDEN-HI/LO, TEMPERED, GRID) | **NOT RUN.** | +| **EV-1 … EV-10** | **ALL TEN NOT RUN.** Every v1 spec was ruled not-sound; the current specs are v2 rewrites awaiting execution. EV-5 is **blocked** — its fixture cannot exist at the pinned timestep. | +| **Five excluded land candidates** in the regime preflight | **UNREPRODUCIBLE.** Their box centres were never recorded anywhere. No coordinates were invented to fake the rows. | +| **W5 at n_idx = 19/21** | **NOT RUN** — iteration count scales ~2V/h², orders of magnitude more expensive. Mechanism stated, not silently dropped. | +| **`ValueTenant::HelixResidue`** | **Zero writers, zero decoders** tree-wide (exhaustive grep, verified twice). The helix-in-substrate story is unwired at the contract boundary. | +| **helix CI** | **No CI gate anywhere** — workspace-excluded, in no workflow. Every helix `[G]` rests on hand-run tests. | +| **BF16 in a prognostic path** | **Untested risk.** No conservation falsifier exists; no error budget quantified. | +| **`E-R²-IS-NEAR-BLIND`** | **Cited twice by id, never written as an entry.** The underlying finding is real (S4 above); the id is not. | +| **Claim C4** (beat a learned model) | **Explicitly out of scope**, quarters of cost. Not attempted, not planned. | +| Out-of-tree performance priors (`~125 ms compute / 233 ms disk`) | **Operator-reported, not in-tree.** Usable as priors, **never as citations**. | + +--- + +## §6 READ-OFF: what to build on, what to stop building on + +**Build on these** — measured, controlled, replicated: + +- the **wn-1 spine** as a compression target (14 values, R² 0.91–0.94); +- **rank-normalisation onto a shared palette256** as the cross-variable + frame (it is the only frame in which the distance is defined); +- **per-class analytic codebooks chosen by measurement**, with Fisher-z + used for **tail reads only**; +- a **global** codebook, never per-storm (139× penalty); +- **tempered coprime strides for bounded budgets**, golden angle for + unbounded ones, with the index floor treated as a hard gate; +- **circular statistics with dual controls** as the directional instrument. + +**Stop building on these** — tested and lost: + +- the **signed** left-of-motion prediction as a scoring referent (P2, and + its steering-level rescue P5); +- the **W6 vector-sum dipole in its current form** (VOID by its own control); +- **Fisher-z as a general L4 axis** (4.7× worse on level reads); +- the **Go-territory tessellation** as specified (both halves, incl. controls); +- **binomial sign fractions** as verdict-grade statistics; +- **naive `round(frac·q)`** strides anywhere. + +**The single most valuable thing this arc produced** is not on either list. +It is the **apparatus**: nine rules in §4, each bought with a measured +failure, which is why a "D" in this document can be trusted as much as an +"A". Most of the entries here are negative results, and they were expensive +to get right — an arc that only reported its wins would have shipped the +36-parameter compression figure, Fisher-z as the universal axis, the +sign-test headline, and a stride-11 "golden" walk.