diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index d079e9d58..5157a30a7 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,127 @@ +## 2026-08-11 — E-THE-FRAME-WAS-ALREADY-SHIPPED-FOUR-TIMES-1 + +**Status:** FINDING `[G]` — read in source this session. + +**The `mu + kσ` calibrate → band → roll-on-drift frame exists FOUR times in this +workspace, and I hand-wrote a fifth in Python after being corrected into it.** + +| instance | metered quantity | +|---|---| +| `ndarray::hpc::cascade::Cascade` — `expose()` → `Band`, `observe`/`recalibrate` | Hamming distance | +| `perturbation_sim::rolling_floor::RollingFloor` — `threshold()` = *"the **confidence-interval** floor"*, `z()` = *"the **Jirak-honest** noise-floor units; significance via `n^(p/2−1)`, not IID"*, `band()` → Stable…Alarm, `preheat()` | mode instability | +| `helix::quantize::RollingFloor` — `quantize`/`bucket_center`/`observe`/`roll` | palette `[a,b]` | +| `thinking-engine::domino` — *"3σ top-K focus"* | table attention | +| **`probes/weather-p1/p1_ci_vs_floor.py` — MINE** | ERA5 bucket CI | + +`perturbation_sim::RollingFloor` **is** the "corrected evaluation frame" that +took three correction sections (§12.10–§12.12) and an operator intervention to +reach — including the Jirak citation, in the doc comment. + +**Also found in the same sweep, all `[G]`:** + +- **`crates/perturbation-sim` is the applied instance of the whole stack** — + `splat.rs` (Gaussian-splat **magnitude** side) + `sketch.rs` (Walsh/XOR **sign** + side) is *literally* OGAR's two-algebra rule; `cascade_key.rs::morton48` is the + OGAR production HHTL address; `hhtl.rs` derives `(HEEL,HIP,TWIG)` by **Cheeger + bisection of the Laplacian**. +- **64×64 "Stockfish ergonomics" is an exact identity:** `64×64 = 4096 cells = + 4096 bit = 512 byte = 64 × u64` = **the CANON node stride**. A node's bits ARE a + bitboard; `masked_popcount_batch(words, mask)` IS `popcount(attacks & targets)`; + magic bitboards are the same LUT amortization as the `[a,b]` floor. +- **`symbiont/src/domino.rs` already proves the AMX path** — 4×4 Morton BF16 + tiles, **16 SoA boards per AMX 16×16 tile GEMM**, cascade feedback, real + `TDPBF16PS` on Emerald Rapids. + +**Rule:** **before writing a frame, grep the workspace for the frame.** The +`Consult, don't guess` ladder (card → knowledge doc → board → source) has no rung +for *"search the sibling crates for the thing you are about to build"* — four +misses in one document say it needs one. A measured number computed twice is not +a wrong number; it is a wasted one, and it hides the fact that the first +implementation already carried the caveats you were about to rediscover. + +## 2026-08-11 — E-TOPOLOGY-PICKS-THE-TABLE-NOT-THE-DOMAIN-1 + +**Status:** FINDING `[G]` — `DistanceLut::circular()` proven a metric +EXHAUSTIVELY over all 256³ = 16 777 216 triples (`distance.rs`, 3 tests). + +**Supersedes both `E-THE-REUSE-IS-THE-PROCESS-…-1` and +`E-JUDGE-THE-FIELD-NOT-THE-ELEMENT-1`.** Operator: *"distance.rs is normalized +[a,b] amortizing in LUT."* + +**The LUT is the AMORTIZATION POINT, not merely a metric or a cache.** +`quantize()` normalizes `[a,b]` once per element at ingest; `from_floor()` folds +the *same* normalization into the table. Afterwards: no bounds, no division, no +per-element normalization — a pure index lookup, in **unit-free** units, which is +exactly what licenses cross-variable comparison. O(256²) once, not O(N²). + +**Consequence:** if the LUT amortizes *any* bounded `[a,b]`, a **circular** range +is just another bounded range with a different formula. Built and proven: +`circular()` = `min(|a−b|, 256−|a−b|)`, the cycle-graph geodesic on `Z_256` — +**0 violations / 16 777 216 triples**, symmetric, identity, positive. Falsifier: +`d_circ(255,0) = 1` vs `d_linear(255,0) = 255`. + +**So the crate's *"raw-azimuth is NOT a metric (the 2π wrap)"* is about the +FORMULA, not about angles.** `linear()` is the wrong table for a ring; +`distance_heuristic` uses no table at all. A wrapping quantity in the 256-palette +with the circular table is metric-safe and stays in the index domain. + +| azimuth as | resolution | metric? | field shape | +|---|---|---|---| +| u16 raw + `linear()` | 0.0055° | **no** | not tileable | +| **u8 palette + `circular()`** | **0.352° mean** | **yes** | `&[u8]` · 128 KB L1 · `U8x64` · AMX | +| nearest-`n` | 0.972° mean | yes | single index | + +The palette azimuth beats nearest-`n` **and** keeps the field ergonomics. The +previous entry was right about the ergonomics and wrong to treat them as +disqualifying: the fix was never "abandon the direct path", it was **give the +wrapping lane its own table**. + +**Rule:** **a bounded quantity's TOPOLOGY selects its table formula; it never +decides whether the quantity belongs in the palette domain.** Amortization, L1 +residency, `U8x64` lane and AMX plane are identical either way — that is what +makes the substrate general rather than per-quantity. + +## 2026-08-11 — E-JUDGE-THE-FIELD-NOT-THE-ELEMENT-1 + +**Status:** FINDING `[G]` (crate-stated: `distance.rs:8-12`; `ndarray` +`int8_tile_gemm::int8_gemm_amx_tiled(a_u8, b_i8, …) -> [i32]`). + +**Corrects `E-THE-REUSE-IS-THE-PROCESS-AND-IT-EXPOSED-A-FIT-PROBLEM-1` (below): +its 10× measurement stands, its VERDICT is inverted.** + +**Operator:** *"you didn't factor in that due to normalized values the field has +different ergonomics than the single value — meaning AMX matmul, tile ops etc."* + +**A normalized representation must be judged by what its FIELD does, not by what +one element decodes to.** The 10× scored **angular reconstruction error** — the +operation the substrate exists to avoid, and the exact metric this workspace +already ruled out for scoring a one-way address over a retained original. Third +instance of that error in one arc; this one landed three sections after the rule +was written down. + +At field scale the ergonomics run the other way: + +| | nearest-`n` | direct `(polar, azimuth)` | +|---|---|---| +| direction collapses to | **ONE index**, 256-palette domain | 3 lanes, one 16-bit **circular** | +| compare | 2 × `DistanceLut` u8 lookups — **L1 metric**, CAKES/CLAM-safe | **not a metric** — `distance.rs:8-10`, the 2π wrap "must never feed CAKES bounds" | +| LUT | 128 KB, L1/L2-resident, **`U8x64`-friendly** | 65536² is not a table | +| tile shape | a `&[u8]` plane → `int8_gemm_amx_tiled` **directly** | none | +| decode to compare? | **no** | **yes** | +| per-point error | 0.972° | 0.097° | + +**Resolution — split by OPERATION, not a winner.** Compare/search/correlate a +field → the single-index path (this is what *"pay the inbound tax once"* buys, +and why palette256 is the same pattern one rank down). Materialize one bearing → +the direct path, 10× finer, but *"never reconstruct per element when the +representation is normalized"* makes that the rare path, not the design centre. + +**Rule:** a per-element accuracy number is the round-trip metric wearing a +different hat. A representation can win it and simultaneously destroy the +index-domain comparison, the metric guarantee, and the tile shape that made the +substrate worth building. **Ask what the field does under the ops you actually +run — LUT, SIMD lane, tile/AMX — before ranking encodes.** + ## 2026-08-11 — E-THE-REUSE-IS-THE-PROCESS-AND-IT-EXPOSED-A-FIT-PROBLEM-1 **Status:** FINDING `[G]` on the measurement (`crates/helix/tests/bearing_encode_paths.rs`, diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index 001af28e1..67ea92024 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -1,3 +1,32 @@ +## 2026-08-11 — weather-substrate-evaluation-v1 (EVALUATION PLAN; the known-vs-test ledger for the whole #915→#922 arc) + +Plan: `.claude/plans/weather-substrate-evaluation-v1.md`. Operator ask: *"create +an evaluation plan with the details known vs what to test."* Two halves. **§1 +KNOWN** — 25 claims (K-1..K-25), each with file:line evidence and an honest +grade split `[G]` / `[G-session]` (probe source committed, fixture fetched) / +`[H]` (measured this session, reproducer NOT committed — only K-12/K-13, the +latitude-band and palette-azimuth tables, whose scratch tests were deleted) / +`[G-absence]`. Every K-row was **independently verified against the tree by a +13-agent verify/attack workflow before the plan went ACTIVE** (7 Sonnet +source-verifiers over helix/quantize/probes/perturbation-sim/ndarray/domino/ +splat3d/morton-comma; 6 Opus falsifier-auditors attacking the EV specs for +vacuous pass routes, missing silence halves, and inert thresholds — the +session's own recurring failure modes, §12.12–§12.17). **§3 TO TEST** — ten +probes EV-1..EV-10 in three waves: Wave 0 needs no data (EV-9 commits the +K-12/K-13 orphans); Wave 1 needs one `fetch.py` re-fetch (EV-1 +advection-as-Morton-shift falsifier · EV-2 wind-lane field-level encode closing +§12.13–§12.15 · EV-3/EV-4 floor & window sweeps feeding operator decisions +D-1/D-2 · EV-5 U-shaped variables, the shape rule's other half · EV-6 harness +re-expression on the SHIPPED `RollingFloor` frame with an exact-number +equivalence gate · EV-8 Jirak effective-n · EV-10 second timestep/season); +Wave 2 is the operator-named 16k×16k 3DGS top-k scale run (EV-7a) + the +comma anti-moiré falsifier (EV-7b, two-sided: the regular-stride control MUST +alias or the fixture cannot falsify). **§5 decision register** D-1..D-6 keeps +the operator calls (noise floor, window, `from_bearing` API, dormant-lane fix, +helix CI, harness-of-record) separate from the tests that feed them. Binding +frame rules restated: bucket-CI vs noise floor never round-trip; Jirak +significance; *grep the workspace for the frame before writing one*. Doc-only. + ## 2026-08-10 — weather-substrate-poc-v2 (PLAN; supersedes v1/#914) Plan: `.claude/plans/weather-substrate-poc-v2.md`. **Supersedes `weather-substrate-poc-v1.md` (#914).** Restructured to the operator's three-phase gate: **A** representation reliability (`jc` battery) → **B** hardware acceleration (`ndarray` parity/throughput) → **C** prediction correctness (`jc` battery). A and C are the SAME instrument on different pairs (`corr(code_dist, field_dist)` vs `corr(predicted, observed)`), so C costs no new statistical machinery; B sits between because a Phase-C number measured on a silently-scalar path would be dishonest about the substrate. **Four corrections to v1 + this plan's own first draft:** (1) **GRIB2 is GONE** — WeatherBench2 publishes `era5/…1440x721.zarr` (= the 1,038,240 grid) on public GCS, so ingest is Zarr→numpy→f32 slab, no eccodes/gribberish, and v1 §6.3 is moot rather than solved; history sizing 58k→**570k** states (65 yr hourly). (2) `ecmwf-opendata` is **Phase C** (live IFS/AIFS), not Phase A (ERA5 reanalysis). (3) **Versioning is CONSUMED, not built** (operator: "lance 900..913++ should have introduced the necessary versioning") — `VersionedGraph::{at_version, current_version, commit_encounter_round}`, `GraphDiff`, `LanceCycleWriter` (#913 +1030), `temporal::{QueryReference::at(v,rung), deinterlace, LanceVersion}` are all shipped and pinned by two existing tests (`a_whole_cycle_of_casts_is_one_wal_write_one_version`, `p4a_drains_…`); **any weather deliverable re-implementing a version writer is the defect.** (4) **S3 is the hydration path, NEVER the store** (#901 doctrine) — the earlier "slabs land on S3, read them back" framing collapsed two layers; object store hydrates, local mmap-capable dir stores, `RAILWAY_VOL` only sets hydration frequency; the network-mount-looks-local trap is called out. **Ingest split** disposable Stage-A (static ERA5, thrown away) vs permanent Stage-C (recurring), sharing the 512 B stride + `soa:*` metadata so Stage-A slabs stay readable. **One dataset, versions are cycles** (operator ruling) — forecast+analysis in the same dataset, joined by version-range read. **All 3 comparison lanes** (forecast-vs-analysis · model-vs-model · encoder-drift; the third guards the substrate claim). **New repos: ZERO** — `crates/weather-poc` workspace-EXCLUDED on the perturbation-sim template; weatherbench2/arco-era5/ecmwf-opendata forks cloned, graphcast zipball-on-demand (under the 3-reads bar). Pins verified against the tree: rust 1.97.1 · lance/lance-encoding/lance-linalg 9.0.0 · lancedb 0.33.0 · arrow 58.3.0 · datafusion 53. Gate D-WXA-5: ρ ≥ 0.98 for ≥1 arm **AND** the shuffled-codebook control must FAIL. Doc-only. diff --git a/.claude/board/PR_ARC_INVENTORY.md b/.claude/board/PR_ARC_INVENTORY.md index 57f523ca4..c9dd62872 100644 --- a/.claude/board/PR_ARC_INVENTORY.md +++ b/.claude/board/PR_ARC_INVENTORY.md @@ -6,7 +6,9 @@ - **Deferred.** **No public `from_bearing` was minted** — deliberately. The measurement says which path is right; the API shape (does the bearing write go through `ResidueEncoder`? where does `rim`'s `n` come from?) is an operator call, `[S]`. Also still open: the saturation-window widening; a citable per-variable noise floor; the U-shaped-variable falsifier. Filed-not-fixed: the `Signed360::sign()` dormant-lane defect. - **Docs.** §12.13; EPIPHANIES `E-THE-REUSE-IS-THE-PROCESS-AND-IT-EXPOSED-A-FIT-PROBLEM-1`, which corrects `E-THE-DOCTRINE-DOC-EXISTED-AND-I-NEVER-READ-IT-1`'s "deliberately not built" framing while keeping its three surviving findings (`Signed360`-is-complete, the `Pair48` withdrawal, the mandatory `READ BY:` rule). -**Confidence (2026-08-11):** merged. The 10× is `[G]` — a committed, re-runnable test, not a one-off. `helix` is excluded from the root workspace and in no CI workflow `[G-absence]`, so it runs **only by hand**. The API shape is `[S]` and must not be built on before an operator decides it. +**Correction (2026-08-11, same session — operator):** *"you didn't factor in that due to normalized values the FIELD has different ergonomics than the single value — AMX matmul, tile ops etc."* **Correct, and it inverts the verdict.** The 10× scored **angular reconstruction error** — the one operation `§12.10` rules out and the substrate exists to avoid. At FIELD scale the ergonomics run the other way: Path A collapses direction to **one index in the 256-palette domain**, so comparison is `rim.distance_adaptive` = two `DistanceLut` u8 lookups — an **L1 metric**, CAKES/CLAM-safe, `U8x64`-friendly (`distance.rs:12`), and a `&[u8]` plane that feeds `ndarray`'s `int8_gemm_amx_tiled(a_u8, b_i8, …)` directly. Path B's azimuth is **16-bit and circular**, which `distance.rs:8-10` names explicitly as **NOT a metric** ("must never feed CAKES bounds") — no 256×256 LUT, no tile plane, and comparison would require decoding. **So: Path A for compare/search/correlate over a field (the substrate's reason-for-being); Path B only where a single bearing must actually be materialized.** The measurement stands; what was wrong was treating it as the deciding metric. + +**Confidence (2026-08-11):** merged. The 10× is `[G]` — a committed, re-runnable test, not a one-off — **but it is not the deciding number; see the Correction above.** `helix` is excluded from the root workspace and in no CI workflow `[G-absence]`, so it runs **only by hand**. The API shape is `[S]` and must not be built on before an operator decides it. ## 2026-08-11 — lance-graph #920 (MERGED) — the probes RAN, and they falsified the doc that queued them: evaluation frame, Fisher-Z economics, and the helix regrade diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 0062c1b10..62af04002 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -1,3 +1,23 @@ +## weather-substrate-evaluation-v1 — EV queue (PRE-REGISTERED 2026-08-11) + +Plan: `.claude/plans/weather-substrate-evaluation-v1.md` (known-vs-test ledger; +verify/attack-audited before ACTIVE). Waves: 0 = no data · 1 = one fixture +re-fetch · 2 = scale. + +| D-id | Deliverable | Wave | Status | Feeds | +|---|---|---|---|---| +| EV-1 | Advection-as-Morton-shift falsifier (wind (dx,dy) tile shift vs persistence, calm-tile silence half) | 1 | Queued | §12.16 [S] regrade | +| EV-2 | Wind-lane encode at FIELD level (nearest-n vs u8-palette-circular vs u16-linear; wrap-corruption can-fire + 90°-sector silence) | 1 | Queued | D-3 | +| EV-3 | Floor-sensitivity sweep [0.1..2.0] K → per-variable flip-points | 1 | Queued | D-1 | +| EV-4 | Saturation-window sweep → sat% vs interior-CI tradeoff curve | 1 | Queued | D-2 | +| EV-5 | U-shaped variables (total_cloud_cover, sea_ice_cover) — the shape rule's OTHER half, two-sided by construction | 1 | Queued | E-TRANSFORM-SHAPE promotion/refutation | +| EV-6 | P1/P2 harness re-expressed on the SHIPPED RollingFloor frame; equivalence gate 0.848/0.820/95.65 + 0.9997/0.999556 to 1e-4 rel | 1 | Queued | D-6 | +| EV-7a | 16k×16k 3DGS top-k scale run (1 048 576 tiles; heel-reject ∉ {0%,100%}) | 2 | Queued | operator-named capability | +| EV-7b | Comma anti-moiré falsifier at tile scale (regular stride MUST alias; comma must not) | 2 | Queued | D-QUANTGATE evidence | +| EV-8 | Jirak effective-n for the P2 correlations (spatial autocorrelation → n_eff) | 1 | Queued | P5 | +| EV-9 | Commit the K-12/K-13 orphan measurements as disable-verified helix tests | 0 | Queued | honesty-split closure | +| EV-10 | Second timestep + season for P1 ([H]→promotion or timestep-conditional regrade) | 1 | Queued | K-9 robustness | + ## oracle-funnel-probe v1 — PROBE-ORACLE-FUNNEL staged (PRE-REGISTERED 2026-08-05) Plan: `.claude/plans/oracle-funnel-probe-v1.md`. Consumes OGAR #241/#244 diff --git a/.claude/knowledge/weather-normalized-substrate.md b/.claude/knowledge/weather-normalized-substrate.md index 059ed604b..805b1567b 100644 --- a/.claude/knowledge/weather-normalized-substrate.md +++ b/.claude/knowledge/weather-normalized-substrate.md @@ -1138,3 +1138,219 @@ the doc's `from_normal`. The doc's prescription is correct for its own case (normals) and should be labelled as such rather than read as universal. **`[S]` until an operator decides the API shape** — this section records the measurement and the tradeoff, and deliberately does not mint a public `from_bearing`. + + +#### 12.14 — ⊘⊘ §12.13's VERDICT IS INVERTED: a normalized FIELD has different ergonomics than a single value + +**Operator (2026-08-11):** *"you didn't factor in that due to normalized values +the field has different ergonomics than the single value — meaning AMX matmul, +tile ops etc."* + +**Correct, and it reverses §12.13's recommendation.** §12.13 scored the two +bearing encodes by **angular reconstruction error** — *the operation §12.10 +rules out and the whole substrate exists to avoid.* Third instance of the same +error in one document, and this one came three sections after writing the rule. + +| | **Path A — nearest `n`** | **Path B — direct `(polar, azimuth)`** | +|---|---|---| +| direction collapses to | **ONE index**, 256-palette domain | 3 lanes, one 16-bit **circular** | +| field comparison | `rim.distance_adaptive` = 2 × `DistanceLut` u8 lookups — **L1 metric**, triangle inequality holds, CAKES/CLAM-safe | azimuth is **NOT a metric** — `distance.rs:8-10`: *"the 2π wrap … must never feed CAKES bounds"* | +| LUT | 256×256×u16 = **128 KB, L1/L2-resident, `U8x64`-friendly** (`distance.rs:12`) | 65536² is not a table | +| tile / AMX shape | a `&[u8]` plane → `ndarray::hpc::int8_tile_gemm::int8_gemm_amx_tiled(a_u8, b_i8, …) → [i32]` **directly** | no single-index plane to tile | +| must decode to compare | **no** | **yes** — the forbidden move | +| per-point angular error | 0.972° | **0.097°** (§12.13) | + +**The resolution is a split by OPERATION, not a winner:** + +- **Compare / search / correlate a field** — Path A. One index, L1-metric LUT, + `U8x64`, AMX-tileable, never decodes. This is what *"pay the inbound tax + once"* actually buys, and it is why palette256 is the pattern one rank down. +- **Materialize one bearing** — Path B, 10× finer. But *"never reconstruct per + element when the representation is normalized"* makes this the rare path, not + the design centre. + +**Rule extracted (the generalizable one):** **judge a normalized +representation by what its FIELD does, not by what one element decodes to.** A +per-element accuracy number is the round-trip metric wearing a different hat — +and a representation that wins it can simultaneously destroy the index-domain +comparison, the metric guarantee, and the tile shape that made the substrate +worth building. §12.13's measurement stands; its *verdict* does not. + +#### 12.15 — RESOLUTION: the LUT amortizes `[a,b]`, so the topology is a TABLE FORMULA, not a disqualification + +**Operator (2026-08-11):** *"distance.rs is normalized [a,b] amortizing in LUT."* + +**That is the load-bearing property, and I had mis-stated it as "a metric/cache".** +The chain, read end to end: + +- `RollingFloor::quantize(v)` = `(v − lo)/(hi − lo)·256` — the `[a,b]` + normalization, **once per element at ingest** (`quantize.rs:99-108`). +- `DistanceLut::from_floor` = `|center(a) − center(b)| / span · 255` — **the same + normalization folded into the TABLE** (`distance.rs:39-50`). + +After the build there is no `lo`, no `hi`, no division, no per-element +normalization — every comparison is a pure index lookup, and the table is +already **unit-free**, which is exactly what licenses cross-variable comparison +(§12.8's 0.9997 shared-palette vs 0.857–0.875 raw cross-unit). **O(256²) once +instead of O(N²) divisions.** This IS *"you only pay the inbound tax once"*, +instantiated. + +**The consequence supersedes BOTH §12.13 and §12.14.** If the LUT amortizes +*any* bounded `[a,b]`, then a **circular** range is just another bounded range +needing a different table formula. Built and proven (`distance.rs::circular`, +`d(a,b) = min(|a−b|, 256−|a−b|)` — the cycle-graph geodesic on `Z_256`): + +| | result | +|---|---| +| triangle inequality | **0 violations / 16 777 216 triples** — EXHAUSTIVE, not sampled | +| symmetry · identity · positivity | all hold, exhaustively | +| the wrap (the falsifier) | `d_circ(255,0) = 1` vs `d_linear(255,0) = 255` | + +**So `distance.rs:8-10`'s *"the raw-azimuth … is NOT a metric (the 2π wrap)"* is +a statement about the FORMULA, not about angles.** `linear()` is simply the wrong +table for a ring; `distance_heuristic` compares raw bytes with no table at all. A +wrapping quantity **quantised into the 256-palette and given the circular table** +is metric-safe and stays in the index domain. + +**Corrected standing:** + +| azimuth as | resolution | metric? | field shape | +|---|---|---|---| +| u16 raw + `linear()` (what §12.13 measured, §12.14 rejected) | 0.0055° | **no** | not tileable | +| **u8 palette + `circular()`** | 1.406° step, **0.352° mean** | **yes**, exhaustively | `&[u8]` plane · 128 KB L1 LUT · `U8x64` · AMX | +| nearest-`n` (§12.13 Path A) | **0.972° mean** | yes | single index | + +**0.352° beats Path A's 0.972° AND keeps the field ergonomics.** §12.14's +inversion was correct about the *ergonomics* and wrong to treat them as +disqualifying — the fix was never "abandon the direct path", it was **give the +wrapping lane its own table**. + +**Rule extracted:** **a bounded quantity's TOPOLOGY selects its table formula; +it never decides whether the quantity belongs in the palette domain.** Linear +range → `linear()`/`from_floor()`. Ring → `circular()`. The amortization, +the 128 KB L1 residency, the `U8x64` lane and the AMX plane are identical either +way — that is what makes the substrate general rather than per-quantity. + +**Shipped:** `DistanceLut::circular()` + 3 tests (exhaustive metric proof; the +wrap falsifier; the remaining axioms). Still `[S]`: whether the wind lane +actually adopts u8-palette azimuth is an ingest-policy call — this section +establishes only that the option is metric-safe and field-shaped. + +#### 12.16 — the `[a,b]` domain is not just comparison: perturbation, Morton cascade, 3DGS, and the comma + +**Operator (2026-08-11):** *"using [a,b] allows also perturbation ergonomics +including testing spatial gaussian splat (3dgs) over 16k×16k topk … or morton +tile cascade … with irrational number pythagorean comma."* + +**Every piece named already exists.** Verified this session — the value of the +normalized domain is not "comparison is cheap", it is that **four separate +capabilities all become the same index-domain operation**: + +| capability | where it lives | verified | +|---|---|---| +| **normalized `[a,b]`, amortized** | `quantize.rs:99` + `distance.rs:39` — normalization folded into the table once; comparisons are pure lookups in unit-free units | `[G]` §12.15 | +| **perturbation phase** | the **discrete Pythagorean comma** — a stride-4-over-17 coprime walk, *"integer, bit-exact, aperiodic"*, **0 stored bits, regenerated from the address alone** (`fire_forget_replay_probe.rs:70-73`). `17 = 4²+1` **IS** the comma: *"without the +1, stride 4 mod 16 covers 4/16"* (`probe_hhtl_intake_blindness.rs:410`) | `[G]` — fence run: `comma_three_gap_distinct=3` (Steinhaus ≤3) and `comma_coprime_full_perm=1` | +| **Morton tile cascade** | `morton_shift_motion_probe.rs` — **motion is O(1) in pixels**: `dx=7 dy=5 bit_exact=1 motion_bytes=2 sprite_px=576`; interior residual **0**, only the disocclusion strip is new (`disocc_frac=0.1215`) | `[G]` — probe run | +| **3DGS** | `ndarray::hpc::splat3d` (feature-gated **directory**, not a file — a first check for `splat3d.rs` wrongly reported it absent): `gaussian`/`project`/`raster`/`tile`/`spd3`/`sh`/`ply`/`depth_cascade`, Kerbl et al. SIGGRAPH 2023 SH layout, `TILE_SIZE = 16`. Its `depth_cascade` is **already HHTL** — `HhtlTier`, `HhtlAction`, `heel_reject_scalar`/`heel_reject_mask` | `[G]` — read | + +At **16 384²** with `TILE_SIZE = 16` that is **1024 × 1024 = 1 048 576 tiles**, +which is the scale at which "stay in the index domain" stops being an +optimization and becomes the only option. + +**Why the comma specifically, and not φ.** In a *quantized* layer a regular tile +grid plus a regular perturbation **aliases** — moiré. The comma walk is the +anti-moiré dither, and it is free: phase is *convention*, not data +(0 stored bits, address-derived). OGAR's `CLAUDE.md` already rules this as +**D-QUANTGATE**: *"in quantized layers the phase generator must be the +coprime-integer walk (helix `CurveRuler` stride-4-over-17, bit-exact integer), +golden recurrence only as build-time muscle-memory; the deterministic phase +doubles as the anti-moiré dither."* `prove.rs` uses `log₂(3/2)` as the *control* +φ beats in the **continuous** 2-D discrepancy proof — a different question in a +different layer, and NOT a demotion of the comma. Board: `E-COMMA-PERTURBATION-PHASE-1` +(named there as *"the Fujifilm X-Trans move"* — X-Trans avoids moiré with an +aperiodic 6×6 CFA instead of an optical low-pass filter; same trick). + +**`[S]` — my synthesis, NOT measured, stated as the next probe:** for weather, +**advection IS a Morton shift.** The `legA` result says a whole field transports +bit-exact for 2 bytes with only an inflow strip as new content — and the thing +that supplies `(dx, dy)` is *the wind bearing*. If that holds, the wind lane is +not merely *encoded* alongside the scalar fields, it is **the transport operator +for them**, and the bearing-encode question (§12.13–§12.15) is really a question +about the shift operator's precision. **Falsifier:** take two consecutive ERA5 +timesteps, derive `(dx, dy)` per tile from the wind field, apply the Morton +shift to `2m_temperature`, and measure the residual against the true next +timestep — versus a no-shift baseline. If the residual does not drop +substantially, advection-as-shift is wrong for real fields at 0.25° / 1 h and +this paragraph must be regraded. + +#### 12.17 — `perturbation-sim` + `domino.rs`: the stack is BUILT, and I re-derived a shipped frame in Python + +**Operator:** *"also check lance-graph / crates / perturbation … morton allows +64×64 stockfish ergonomics … also check domino.rs."* All three land. Read, not +inferred: + +**(a) `crates/perturbation-sim` is the APPLIED instance of this entire thread** — +an outage cascade simulator that already composes every piece §12.16 named: + +| module | what it already is | +|---|---| +| `rolling_floor.rs` | *"L1..L4 HHTL tiers as an HDR popcount-stacking, early-exit, **Belichtungsmesser** cascade with **preheated confidence-interval thresholds**"* | +| `splat.rs` | *"Gaussian-splat **magnitude** side of the pyramid… the **sign** side of the Morton pyramid is Walsh/XOR (`sketch`)"* — with `morton2`, `box_coarsen`, `ewa_coarsen` | +| `cascade_key.rs` | *"Full 16-bit-per-tier spatial cascade key — the OGAR production form of the HHTL address"* — `from_spectral`, `to_guid_tiers`, **`morton48`**, `cascade_distance`, `tile` | +| `hhtl.rs` | HHTL `(HEEL,HIP,TWIG)` by **recursive Cheeger bisection of the Laplacian** — the address is derived spectrally | +| `columns.rs` | `SoaMemberSpec` — the #511 calibration | + +The two-algebra rule (OGAR `CLAUDE.md`: sign = XOR, magnitude = bundle, **never** +`MergeMode::Xor` on magnitudes) is **literally the `splat`/`sketch` split here.** + +**(b) ⊘ I re-derived `RollingFloor` in Python `[G]` — the most expensive miss of +this arc.** `perturbation_sim::rolling_floor` already ships, with doc comments: + +- `threshold()` = **`mu + k·σ`** — *"The **confidence-interval** floor"* +- `z(x)` = `(x−mu)/σ` — *"the **Jirak-honest** 'noise-floor units'; significance + via `n^(p/2−1)`, **not IID**"* +- `band(x)` → Stable/Watch/Concern/Warning/**Alarm**; `preheat()`; `observe()` + tests against the floor **as it stood**, then updates (the rolling property) + +**That IS §12.10/§12.11's "corrected evaluation frame", verbatim, Jirak citation +included** — and I rebuilt it by hand in `probes/weather-p1/p1_ci_vs_floor.py` +after an operator correction. It is not even the only instance: the same +`mu + kσ` → band → roll-on-drift frame appears **four** times — +`ndarray::hpc::cascade::Cascade` (`expose()`, Hamming-metered), +`perturbation_sim::RollingFloor` (instability-metered), +`helix::quantize::RollingFloor` (the palette floor), and +`thinking-engine::domino`'s *"3σ top-K focus"*. The weather probe should be the +**fifth instance of one frame**, not a fifth implementation. + +**(c) 64×64 "Stockfish ergonomics" is an EXACT identity `[G]`:** +`64 × 64 = 4096 cells = 4096 bit = 512 byte = 64 × u64` — **precisely the CANON +node stride** (`key(16) | edges(16) | value(480)`). A Morton 64×64 tile is a +12-bit code (6+6), and a node's worth of bits *is* a bitboard. The bit-parallel +surface is already there: `masked_popcount_batch(words, mask)` — mask a plane, +count — **is** the Stockfish primitive (`popcount(attacks & targets)`), alongside +`popcount_per_word`, `hamming_top_k_raw`, `xor_popcount`, `nibble_popcount_lut`. +The magic-bitboard idea (precomputed table indexed by an occupancy mask) is the +same LUT amortization as §12.15. + +**(d) `symbiont/src/domino.rs` is the working AMX proof `[G]`:** *"BF16 + AMX, +2bit×2bit 4×4 Morton tile… each SoA board carries a 4×4 BF16 tile (16 lanes, +Morton-addressed)… **16 boards batch into ONE AMX 16×16 tile GEMM**… C +re-quantised back into the tiles (cascade feedback)"*, on real Emerald Rapids +with `TDPBF16PS` firing. It also states the consumer rule verbatim: **all SIMD +through `ndarray::simd::*`, never `ndarray::hpc::*`**, and notes `morton4` is +consumer-side because *"ndarray has no Morton primitive"*. + +**Consequence for the weather work — concrete, not architectural:** the P1/P2 +probes should be **re-expressed against the shipped frame** (a `RollingFloor` +consumer + `CascadeKey`/`morton48` addressing + `splat`/`sketch` for the +magnitude/sign split), not carried as bespoke Python. That is a rewrite of the +probe harness, not of any finding: the measured numbers (§12.8, §12.11) stand +because they were computed correctly — they were just computed **twice**. + +**Rule extracted (fourth instance of one lesson in this document):** *before +writing a frame, grep the workspace for the frame.* §12.12 was the doctrine doc +never opened; §12.14 the field ergonomics never checked; §12.15 the table formula +never questioned; this is an **implemented, documented, Jirak-cited frame +re-written in another language**. The `Consult, don't guess` rule in `CLAUDE.md` +orders card → knowledge doc → board → source; nothing in that ladder says *grep +the sibling crates for the thing you are about to build* — and it should. diff --git a/.claude/plans/weather-substrate-evaluation-v1.md b/.claude/plans/weather-substrate-evaluation-v1.md new file mode 100644 index 000000000..4dcf08486 --- /dev/null +++ b/.claude/plans/weather-substrate-evaluation-v1.md @@ -0,0 +1,244 @@ +# Evaluation Plan — Weather Normalized Substrate: KNOWN vs TO TEST (v1) + +> **READ BY:** certification-officer, truth-architect, family-codec-smith, +> integration-lead, measurement-skeptic-analog sessions, and ANY session +> executing an EV-* probe below. +> +> **Status:** DRAFT pending the verify/attack audit pass (§8) → then ACTIVE. +> **Provenance:** the #915/#917/#920/#921/#922 arc + the post-#920 session +> corrections (`weather-normalized-substrate.md` §12.1–§12.17). +> **Grading:** `[G]` verified in committed code/test with file:line · +> `[G-session]` measured this session, re-runnable from committed artifacts · +> `[H]` measured this session, NOT independently re-runnable as committed · +> `[S]` proposal/conjecture — do not build on before its probe runs. + +--- + +## 0. Purpose + +Convert the session's findings into a gated evaluation: one ledger of what is +KNOWN (each entry with its evidence and its honest grade), one queue of what is +TO TEST (each with method, pass/fail, and the falsifier discipline the +workspace mandates: can-it-fire + can-it-stay-silent + no inert thresholds + +never a round-trip metric). + +Binding process rules for every EV-* execution: +- **The evaluation frame** is bucket-CI vs noise floor — never decoded + round-trip error (§12.10/§12.11). +- **Significance** on pooled/spatial data uses Jirak 2016 weak-dependence + rates, never classical IID Berry–Esseen (`I-NOISE-FLOOR-JIRAK`). +- **Before writing a frame, grep the workspace for the frame** (§12.17): the + `mu + k·σ` calibrate→band→roll frame ships 4× already. +- Fixtures: fetched, never committed; probes re-runnable; results JSONs + committed; the `/tmp`-fixture time-bomb rule applies. + +--- + +## 1. KNOWN — the verified ledger + +### 1a. Helix codec facts + +| ID | Claim | Evidence | Grade | +|---|---|---|---| +| K-1 | `Signed360` = 6 B, wire `[rim.start, rim.end, rim.floor_version, polar, az_lo, az_hi]`; ONE complete full-sphere direction | `crates/helix/src/residue.rs:76–116`; doctrine `helix-cartesian-vs-fisher2z.md` §"Signed360 specifics" | [G] | +| K-2 | Sign partition exact at every magnitude: `Pos ⇒ polar ∈ [128,255]`, `Neg ⇒ [0,127]`, incl. the rim `\|y\| ≈ 0` (#498) | `residue.rs:182–204`; tests `signed360_neg_sign_survives_near_rim_at_high_total`, `polar_partitions_are_exactly_the_two_halves` (disable-verified) | [G] | +| K-3 | `azimuth` spans the FULL u16 circle: min 0, max 65535, 256/256 coarse arcs, 54 319 distinct at N=65536 | `crates/helix/tests/signed360_claims.rs::azimuth_spans_the_full_circle_not_merely_varies` (disable-verified: 10-bit truncation → red) | [G] | +| K-4 | Dormant-lane defect: all-zero lane decodes as definite `Sign::Neg` — pinned as a DEFECT test; filed, NOT fixed | `signed360_claims.rs::dormant_all_zero_lane_decodes_as_a_definite_sign_known_defect` | [G] | +| K-5 | `DistanceLut::circular()` = `min(\|a−b\|, 256−\|a−b\|)` is a metric — EXHAUSTIVE 256³ = 16 777 216 triples, 0 violations; wrap falsifier `d(255,0)=1` vs linear 255 | `crates/helix/src/distance.rs::circular` + 3 tests | [G] | +| K-6 | The `[a,b]` amortization: `quantize()` normalizes once at ingest; `from_floor()` folds the SAME normalization into the table → unit-free pure-lookup comparisons | `quantize.rs:99–108`, `distance.rs:39–50` | [G] | +| K-7 | Bearing-encode paths measured, N=65536: nearest-`n` mean 0.972° vs direct `(polar,azimuth)` 0.097° (10×); mechanism = one index couples lat+az, `sin(2·lat)` disk-lattice density | `crates/helix/tests/bearing_encode_paths.rs` (committed, re-runnable) | [G] | +| K-8 | `helix` is root-workspace-EXCLUDED and named in NO CI workflow → all its tests run only by hand | root `Cargo.toml` `exclude`; grep of `.github/workflows/` | [G-absence] | + +### 1b. Weather measurements (ERA5, one timestep 2021-06-15 12:00 UTC unless noted) + +| ID | Claim | Evidence | Grade | +|---|---|---|---| +| K-9 | Fisher-Z on `2m_temperature` anomalies: at 0.5–1 K floor an address-economy failure only (sat 0.848 % linear / 0.820 % fisher-z, otherwise indistinguishable); at 0.25 K also a validity failure (+95.65 % interior-CI exceedance) | `probes/weather-p1/p1_ci_vs_floor.{py,json}`; internally consistent without fixture (linear uniform to 3.8e-14; monotone floor sweep) | [G-session] | +| K-10 | Standardization, not Fisher-Z, licenses cross-variable comparison: 0.9997 shared palette vs 0.857–0.875 raw cross-unit; T×Td shared = 0.999556 (BELOW a 0.9996 bar — rounding once hid this) | `probes/weather-p1/p2_probe.py`, `p2_results.json` | [G-session] | +| K-11 | Gate-1 reliability ran in `jc` (Pearson/Spearman/Cronbach α/ICC) with a `--shuffle` negative control and header validation | `crates/jc/examples/weather_substrate_reliability.rs` + committed `jc_input.bin` | [G] | +| K-12 | `Signed360` angular error by latitude: equator (0–5°) 0.112° mean / 0.226° max; pole (85–90°) 3.332° / 4.998° (~30× spread); NO equal-budget gain from the sign split (7-bit+sign vs 8-bit: 0.99–1.02× every band) | measured this session; recorded in `LATEST_STATE.md` #920 entry — **scratch test was deleted, NOT committed** | [H] | +| K-13 | u8-palette azimuth under `circular()`: 1.406° step, 0.352° mean — beats nearest-`n` 0.972° AND keeps field ergonomics | derived arithmetically + circular-LUT proof; per-band measurement not committed | [H] | + +### 1c. The shipped frame (what the probes must consume, not re-implement) + +| ID | Claim | Evidence | Grade | +|---|---|---|---| +| K-14 | `perturbation_sim::rolling_floor::RollingFloor` IS the corrected evaluation frame: `threshold()` = "the confidence-interval floor" `mu+k·σ`; `z()` = "the Jirak-honest noise-floor units; significance via n^(p/2−1), not IID"; `band()` → Stable…Alarm; `preheat()`; `observe()` tests against the floor as it stood | `crates/perturbation-sim/src/rolling_floor.rs` (~:93–145) | [G] | +| K-15 | `splat.rs` = Gaussian-splat MAGNITUDE side; `sketch.rs` = Walsh/XOR SIGN side — the two-algebra rule instantiated; `morton2`, `box_coarsen`, `ewa_coarsen` | `crates/perturbation-sim/src/{splat,sketch}.rs` | [G] | +| K-16 | `cascade_key.rs`: 16-bit-per-tier OGAR-form HHTL address — `from_spectral`, `to_guid_tiers`, `morton48`, `cascade_distance`, `tile` | `crates/perturbation-sim/src/cascade_key.rs` | [G] | +| K-17 | `hhtl.rs` derives `(HEEL,HIP,TWIG)` by recursive Cheeger bisection of the Laplacian | `crates/perturbation-sim/src/hhtl.rs` | [G] | +| K-18 | `ndarray::hpc::cascade::Cascade` = the Belichtungsmesser original: `calibrate`/`expose`→`Band`/`observe`→`ShiftAlert`/`recalibrate` | `ndarray/src/hpc/cascade.rs` | [G] | +| K-19 | `symbiont/src/domino.rs` proves the AMX path: 4×4 Morton BF16 tiles, 16 SoA boards per AMX 16×16 tile GEMM, cascade feedback, real `TDPBF16PS` (Emerald Rapids); ALL SIMD via `ndarray::simd::*`; "ndarray has no Morton primitive" (consumer-side `morton4`) | `crates/symbiont/src/domino.rs` header + `run_poc` | [G] | +| K-20 | `ndarray::hpc::splat3d` is a full 3DGS pipeline (feature-gated DIRECTORY): gaussian/project/raster/tile/spd3/sh/ply/depth_cascade; `TILE_SIZE = 16`; `depth_cascade` is ALREADY HHTL (`HhtlTier`, `HhtlAction`, `heel_reject_mask`) | `ndarray/src/hpc/splat3d/` | [G] | +| K-21 | Morton motion is O(1) in pixels: `dx=7 dy=5 bit_exact=1 motion_bytes=2 sprite_px=576`; interior residual 0; only the disocclusion strip new (`disocc_frac=0.1215`); comma fence holds (`three_gap_distinct=3`, `coprime_full_perm=1`) | `crates/helix/examples/morton_shift_motion_probe.rs` — RUN this session | [G-session] | +| K-22 | The perturbation phase is the DISCRETE Pythagorean comma: stride-4-over-17 coprime walk, integer, bit-exact, aperiodic, **0 stored bits** (address-derived); `17 = 4²+1` IS the comma; D-QUANTGATE rules it mandatory in quantized layers | `fire_forget_replay_probe.rs:70–73`, `probe_hhtl_intake_blindness.rs:410`, OGAR `CLAUDE.md` D-QUANTGATE | [G] | +| K-23 | 64×64 = 4096 cells = 4096 bit = 512 B = 64×u64 = the CANON node stride; `masked_popcount_batch(words, mask)` IS the Stockfish primitive; magic bitboards = the same LUT amortization | arithmetic identity + `ndarray/src/hpc/bitwise.rs` | [G] | +| K-24 | AMX eats `&[u8]` planes directly: `int8_gemm_amx_tiled(a_u8: &[u8], b_i8: &[i8], c: &mut [i32], …)` | `ndarray/src/hpc/int8_tile_gemm.rs` | [G] | +| K-25 | `ValueTenant::HelixResidue` = `U8×6 @ row_offset 112`; **zero writers, zero decoders** in the whole tree; no per-value-lane reading selector exists in the contract | contract `canonical_node.rs`; exhaustive grep (15 hits) | [G-absence] | + +--- + +## 2. Honesty split — session-only evidence that needs committing + +- **K-12 / K-13 are the only KNOWN rows without a committed reproducer.** The + latitude-band table and the palette-azimuth per-band numbers were measured in + scratch tests that were deliberately deleted. EV-9 commits them. +- K-9/K-10/K-21 are `[G-session]`: the probe SOURCE is committed and re-runnable, + but the fixture is fetched (container-reset-cleared) and the run happened this + session. Re-running is one `fetch.py` away — that is by design, not a gap. + +--- + +## 3. TO TEST — the EV queue + +Every EV lands as a committed, re-runnable artifact (Rust test/example, or a +probe script + result JSON), with its pass criterion stated BEFORE the run. +Bars marked *(provisional pin)* are set relative to a measured baseline in the +same run — never an absolute magic number; first run pins them, and a pinned +bar must then be two-sided. + +### EV-1 — Advection IS a Morton shift (the §12.16 [S] falsifier) +- **Claim under test:** a wind-derived per-tile `(dx,dy)` Morton shift of + `2m_temperature(t)` predicts `t+1h` better than persistence. +- **Method:** 2 consecutive ERA5 timesteps; per tile (16×16 first, 64×64 + control) `(dx,dy) = round(mean 10m wind · 3600 s / 27.8 km)`; shift; residual + vs truth. At 0.25°/1 h, 10 m/s ≈ 1.3 cells — measurable, not sub-cell noise. +- **Pass (can-fire):** on tiles with `|wind| ≥ v_min`, shifted residual MAE + beats persistence MAE *(provisional pin: report the ratio; adopt only if + < 0.8)*. **Silence:** on calm tiles (`|wind| < v_min/2`) shift ≈ persistence + (ratio ∈ [0.9, 1.1]) — a shift that "wins" on calm air is an apparatus bug. +- **Fail →** §12.16's advection paragraph regraded; the wind lane stays a field, + not a transport operator. +- **Inertness:** `v_min` must be non-inert — sweep it; if the pass/fail verdict + is insensitive to `v_min` over [2, 10] m/s, the tile-mean wind is too smooth + and the probe must move to 64×64 tiles. + +### EV-2 — Wind lane encode at FIELD level (closes §12.13–§12.15) +- **Claim:** u8-palette azimuth + `circular()` preserves the angular-distance + structure of a REAL wind field; `linear()` measurably corrupts it. +- **Method:** ERA5 10m u/v → bearings → three encodes (nearest-`n`; + u8-palette-circular; u16-linear). For sampled point pairs: true angular + difference vs table distance. Spearman ρ per encode (Jirak-cited + significance), plus a wrap-corruption COUNT for `linear()` (pairs straddling + 0°/360° whose rank inverts). +- **Pass (can-fire):** palette-circular ρ within 0.02 of the u16 ground-truth + ranking AND `linear()` wrap-corruption count > 0 on real data (the defect + must actually fire on nature, not just on synthetic 359°/1°). + **Silence:** restricted to pairs within a 90° sector (no wrap), `linear()` ≈ + `circular()` — proves the corruption is THE WRAP, not table noise. +- **Fail →** the palette-azimuth option is not field-adequate; §12.15's [S] + adoption question closes NO. + +### EV-3 — Floor-sensitivity sweep (feeds operator decision D-1) +- **Claim:** every floor-dependent verdict (the 0.25 K flip) is stable inside + the plausible obs-error range. +- **Method:** re-run `p1_ci_vs_floor` over a floor grid [0.1 … 2.0] K per + variable; deliverable = the flip-point (floor at which Fisher-Z's interior-CI + exceedance crosses 1 %) per variable. +- **Pass:** flip-points are sharp (not straddling the whole range) and reported + with the sat-% column; the OUTPUT is the decision input for D-1, so the pass + bar is completeness, not a verdict. +- **Vacuity guard:** the sweep must show BOTH regimes (some floor where + Fisher-Z fails, some where it doesn't) — a sweep entirely in one regime + cannot locate a flip-point and must widen its grid. + +### EV-4 — Saturation-window widening (feeds D-2) +- **Method:** sweep the percentile window {0.4–99.6, 0.2–99.8, 0.1–99.9, + 0.02–99.98}; per window: sat-%, interior-CI median/max vs the K-9 floors. +- **Pass:** the tradeoff curve is monotone in the expected directions (sat ↓, + interior CI ↑ as the window widens) — a non-monotone curve means the + apparatus is wrong, not the physics. Decision itself is D-2. + +### EV-5 — U-shaped variables: the shape rule's OTHER half +- **Claim (from `E-THE-TRANSFORM-MUST-MATCH-THE-DISTRIBUTION-SHAPE-1` [S]):** + Fisher-Z should WIN on bound-massed variables (`total_cloud_cover`, + `sea_ice_cover`). +- **Method:** identical P1 pipeline on both variables. +- **Pass (two-sided by construction):** Fisher-Z shows better bucket economy / + interior CI than linear on these — while K-9 already shows the OPPOSITE on + temperature. Either outcome is informative: win → the shape rule promotes + [S]→[H]; loss → the rule is falsified and the epiphany regrades. + +### EV-6 — Harness re-expression on the shipped frame (retires the Python) +- **Claim:** the P1/P2 pipeline expressed as consumers of the SHIPPED frame + (`perturbation_sim::RollingFloor` or `helix::RollingFloor` + `jc` + reliability + `CascadeKey`/`morton48` addressing) reproduces the committed + numbers. +- **Equivalence gate (exact, not approximate):** 0.848 % / 0.820 % / 95.65 % + (P1) and 0.9997 / 0.857–0.875 incl. T×Td = 0.999556 (P2) to 1e-4 relative + from the same fixture. Python stays as provenance (the repo's + `p1_noise_floor.py` pattern), Rust becomes the measurement of record. +- **Vacuity guard:** the Rust harness must NOT share code with the Python + (independent implementation is the point); assert on the committed JSON + values, not on freshly-computed Python output. + +### EV-7a — 16k×16k 3DGS top-k scale run (operator-named) +- **Claim:** the shipped `splat3d` pipeline + HHTL depth cascade handles a + 16 384² canvas = 1024×1024 = 1 048 576 tiles with top-k selection. +- **Method:** synthetic-or-ERA5-derived gaussian population; `TileBinning` at + 1M tiles; `cascade_blocks` + `heel_reject_mask` prune rates; + top-k via `hamming_top_k_raw`/cascade bands. +- **Pass:** completes within a recorded time/memory envelope *(provisional + pin — first run records, second run gates)*; heel-reject rate is neither 0 % + nor 100 % (a pruner that fires on everything or nothing is the + `closed_class_guess` defect). +- **NOTE:** `splat3d` is feature-gated — the run must state the feature flags + and host (AMX availability changes the path). + +### EV-7b — Comma anti-moiré at tile scale (the D-QUANTGATE falsifier) +- **Claim:** comma-phase (stride-4-over-17) perturbation shows no grid-locked + aliasing where a REGULAR stride does. +- **Method:** perturb a uniform field per tile with (a) regular phase + (`addr % k`), (b) comma phase; measure grid-alignment energy (spectral peak + at the tile frequency, or lag-k autocorrelation at tile pitch — NOT plain + lag-autocorrelation, which `mu_hydration_probe` documents as the WRONG + instrument). +- **Pass (two-sided, mandatory):** regular stride MUST show the peak + (can-fire) AND comma must not (silence). If the regular control does not + alias, the fixture cannot falsify and must be re-scaled. + +### EV-8 — Jirak effective-n for the P2 correlations +- **Claim:** the P2 correlations survive weak-dependence-honest significance. +- **Method:** estimate spatial autocorrelation length of the anomaly fields → + effective n ≪ 1 038 240 → re-state P2 significance at Jirak's `n^(p/2−1)`; + feeds P5 (`drift_sigma` under autocorrelation). +- **Pass:** stated effective-n with the derivation; correlations re-graded + against it (expected to survive — but "expected" is not evidence). + +### EV-9 — Commit the K-12/K-13 measurements (Wave 0 — no data needed) +- Re-create the latitude-band error test and the palette-azimuth per-band + measurement as committed `crates/helix/tests/` with tolerance assertions on + the recorded numbers (0.112°/3.332°/0.99–1.02×/0.352°), disable-verified. +- **Pass:** green + each assertion red under an injected defect. + +### EV-10 — Second timestep + season for P1 ([H]→ promotion path) +- **Method:** winter timestep (2021-01-15 12:00 UTC) + one more variable + through the full P1-CI pipeline. +- **Pass:** same qualitative ordering as K-9 (economy failure at plausible + floors). **Fail →** `E-THE-TRANSFORM-MUST-MATCH-…-1` regrades; every §12.11 + conclusion becomes timestep-conditional. + +--- + +## 4. Waves + +| Wave | EVs | Prerequisite | +|---|---|---| +| 0 (now, no data) | EV-9 | none | +| 1 (one `fetch.py` re-fetch) | EV-1, EV-2, EV-3, EV-4, EV-5, EV-8, EV-10, EV-6 | fixture on disk | +| 2 (scale) | EV-7a, EV-7b | feature flags + host statement | + +Per the workspace probe discipline: **if an EV is NOT RUN, the next deliverable +is the probe, not more synthesis.** + +## 5. Operator decision register (open calls; tests feed them, never decide them) + +| ID | Decision | Fed by | +|---|---|---| +| D-1 | Pin a citable per-variable noise floor | EV-3 | +| D-2 | Saturation-window policy | EV-4 | +| D-3 | `from_bearing` / wind-lane API shape (incl. u8-palette azimuth adoption) | EV-2, K-5, K-7 | +| D-4 | Dormant-lane fix shape (`Option` vs sentinel) — the pinned defect test must break deliberately | K-4 | +| D-5 | helix CI wiring (workspace-excluded, no gate anywhere) | K-8 | +| D-6 | Harness language ruling — adopt EV-6's Rust as measurement-of-record | EV-6 | + diff --git a/crates/helix/src/distance.rs b/crates/helix/src/distance.rs index 0c8160629..ca04072ae 100644 --- a/crates/helix/src/distance.rs +++ b/crates/helix/src/distance.rs @@ -49,6 +49,36 @@ impl DistanceLut { Self { table } } + /// **Circular** (ring) order table: `distance(a, b) = min(|a − b|, 256 − |a − b|)` + /// — the cycle-graph geodesic on `Z_256`. Also a metric (verified exhaustively + /// over all 256³ triples by `circular_satisfies_triangle_inequality`), so it + /// IS safe for CAKES / CLAM pruning bounds. + /// + /// **Use this for any quantity whose range WRAPS** — a bearing, a phase, an + /// angle — where [`Self::linear`] is simply the wrong table: it puts index + /// 255 and index 0 at *maximum* distance when they are in fact adjacent + /// (`d_linear(255,0) = 255` vs `d_circular(255,0) = 1`). That failure, not + /// any property of angles themselves, is what the module note above means by + /// *"the raw-azimuth angular pre-filter is NOT a metric (the 2π wrap)"* — + /// [`crate::ResidueEdge::distance_heuristic`] compares raw bytes with no + /// table at all. A wrapping quantity **quantised into the 256-palette and + /// given THIS table** is metric-safe and stays in the index domain. + /// + /// Same amortization as every other constructor: the `[a, b]` normalization + /// is folded in ONCE at build time, so each subsequent comparison is a pure + /// index lookup — no bounds, no division, no per-element normalization. + pub fn circular() -> Self { + let mut table = vec![0u16; PALETTE_SIZE * PALETTE_SIZE]; + let n = PALETTE_SIZE as i32; + for a in 0..PALETTE_SIZE { + for b in 0..PALETTE_SIZE { + let raw = (a as i32 - b as i32).abs(); + table[a * PALETTE_SIZE + b] = raw.min(n - raw) as u16; + } + } + Self { table } + } + /// O(1) metric-safe distance lookup. #[inline] pub fn distance(&self, a: u8, b: u8) -> u16 { @@ -83,6 +113,64 @@ mod tests { } } + /// The wrap is the whole point: on a ring, 255 and 0 are ADJACENT. + /// `linear()` calls them maximally distant — this is the concrete failure + /// the module note warns about, and the reason a bearing needs `circular()`. + #[test] + fn circular_wrap_is_adjacent_where_linear_is_maximal() { + let circ = DistanceLut::circular(); + let lin = DistanceLut::linear(); + assert_eq!(circ.distance(255, 0), 1, "255 and 0 are adjacent on a ring"); + assert_eq!( + lin.distance(255, 0), + 255, + "...but maximal under the linear table" + ); + // and the far side of the ring is the true maximum, at half a turn + assert_eq!(circ.distance(0, 128), 128); + assert_eq!(circ.distance(64, 192), 128); + } + + /// EXHAUSTIVE metric proof — all 256³ = 16 777 216 triples, not a stride + /// sample. A circular table is cheap enough to prove outright, and its + /// metric-safety is exactly the claim that licenses CAKES/CLAM pruning. + #[test] + fn circular_satisfies_triangle_inequality() { + let lut = DistanceLut::circular(); + let mut violations = 0u64; + for a in 0..PALETTE_SIZE { + for b in 0..PALETTE_SIZE { + let dab = lut.distance(a as u8, b as u8) as u32; + for c in 0..PALETTE_SIZE { + let dac = lut.distance(a as u8, c as u8) as u32; + let dbc = lut.distance(b as u8, c as u8) as u32; + if dac > dab + dbc { + violations += 1; + } + } + } + } + assert_eq!(violations, 0, "circular table must be a metric"); + } + + /// Symmetry, identity, and positivity — the rest of the metric axioms. + #[test] + fn circular_is_symmetric_with_zero_diagonal_and_positive_off_diagonal() { + let lut = DistanceLut::circular(); + for a in 0..PALETTE_SIZE { + assert_eq!(lut.distance(a as u8, a as u8), 0); + for b in 0..PALETTE_SIZE { + assert_eq!( + lut.distance(a as u8, b as u8), + lut.distance(b as u8, a as u8) + ); + if a != b { + assert!(lut.distance(a as u8, b as u8) > 0, "d(a,b)>0 for a!=b"); + } + } + } + } + #[test] fn linear_satisfies_triangle_inequality() { // Metric-safety regression (mirrors bgz17's test_palette_triangle_inequality): diff --git a/crates/helix/tests/bearing_encode_paths.rs b/crates/helix/tests/bearing_encode_paths.rs index 070db09fa..c07944833 100644 --- a/crates/helix/tests/bearing_encode_paths.rs +++ b/crates/helix/tests/bearing_encode_paths.rs @@ -24,18 +24,25 @@ use helix::placement::{HemispherePoint, Sign}; use helix::residue::{ResidueEncoder, Signed360}; -fn dir(bearing_deg: f64, elev_deg: f64) -> (f64,f64,f64) { - let (b,e) = (bearing_deg.to_radians(), elev_deg.to_radians()); - (e.cos()*b.sin(), e.cos()*b.cos(), e.sin()) // (x=east, z=north, y=up) +fn dir(bearing_deg: f64, elev_deg: f64) -> (f64, f64, f64) { + let (b, e) = (bearing_deg.to_radians(), elev_deg.to_radians()); + (e.cos() * b.sin(), e.cos() * b.cos(), e.sin()) // (x=east, z=north, y=up) } -fn ang(a:(f64,f64,f64), b:(f64,f64,f64)) -> f64 { - (a.0*b.0+a.1*b.1+a.2*b.2).clamp(-1.0,1.0).acos().to_degrees() +fn ang(a: (f64, f64, f64), b: (f64, f64, f64)) -> f64 { + (a.0 * b.0 + a.1 * b.1 + a.2 * b.2) + .clamp(-1.0, 1.0) + .acos() + .to_degrees() } -fn decode(s: &Signed360) -> (f64,f64,f64) { - let y = if s.polar>=128 {(s.polar as f64-128.0)/127.0} else {-((127.0-s.polar as f64)/127.0)}; - let r = (1.0-y*y).max(0.0).sqrt(); - let a = s.azimuth as f64/65536.0*std::f64::consts::TAU; - (r*a.sin(), r*a.cos(), y) +fn decode(s: &Signed360) -> (f64, f64, f64) { + let y = if s.polar >= 128 { + (s.polar as f64 - 128.0) / 127.0 + } else { + -((127.0 - s.polar as f64) / 127.0) + }; + let r = (1.0 - y * y).max(0.0).sqrt(); + let a = s.azimuth as f64 / 65536.0 * std::f64::consts::TAU; + (r * a.sin(), r * a.cos(), y) } #[test] @@ -44,27 +51,31 @@ fn measure() { let enc = ResidueEncoder::new(N); // Path A (doctrine): nearest spherical-Fibonacci (n, sign), then encode_signed. - let nearest = |target:(f64,f64,f64)| -> (usize, Sign) { + let nearest = |target: (f64, f64, f64)| -> (usize, Sign) { let mut best = (f64::MAX, 0usize, Sign::Pos); for n in 0..N { for sg in [Sign::Pos, Sign::Neg] { let p = HemispherePoint::signed_lift(n, N, sg); - let v = (p.r*p.azimuth.sin(), p.r*p.azimuth.cos(), p.y); + let v = (p.r * p.azimuth.sin(), p.r * p.azimuth.cos(), p.y); let e = ang(target, v); - if e < best.0 { best = (e, n, sg); } + if e < best.0 { + best = (e, n, sg); + } } } (best.1, best.2) }; // Path B: write (polar, azimuth) directly from the bearing; rim still from (place, n). - let direct = |target:(f64,f64,f64), rim_n: usize| -> Signed360 { + let direct = |target: (f64, f64, f64), rim_n: usize| -> Signed360 { let y = target.2; - let mag = (y.abs()*127.0).round().clamp(0.0,127.0) as u8; - let polar = if y >= 0.0 { 128+mag } else { 127-mag }; + let mag = (y.abs() * 127.0).round().clamp(0.0, 127.0) as u8; + let polar = if y >= 0.0 { 128 + mag } else { 127 - mag }; let az = target.0.atan2(target.1).rem_euclid(std::f64::consts::TAU); - let azimuth = ((az/std::f64::consts::TAU)*65536.0).round() as u64 as u16; - let mut s = enc.encode_signed(0x1234, rim_n, if y>=0.0 {Sign::Pos} else {Sign::Neg}); - s.polar = polar; s.azimuth = azimuth; s + let azimuth = ((az / std::f64::consts::TAU) * 65536.0).round() as u64 as u16; + let mut s = enc.encode_signed(0x1234, rim_n, if y >= 0.0 { Sign::Pos } else { Sign::Neg }); + s.polar = polar; + s.azimuth = azimuth; + s }; println!("\nwind bearings (elevation 0 = horizontal), N={N}"); @@ -77,8 +88,12 @@ fn measure() { let (n, sg) = nearest(t); let ea = ang(t, decode(&enc.encode_signed(0x1234, n, sg))); let eb = ang(t, decode(&direct(t, n))); - if elev == 0.0 { println!(" {bear:>6.1}° {elev:>4.0}° {ea:>8.4}° {eb:>8.4}°"); } - sa += ea; sb += eb; c += 1; + if elev == 0.0 { + println!(" {bear:>6.1}° {elev:>4.0}° {ea:>8.4}° {eb:>8.4}°"); + } + sa += ea; + sb += eb; + c += 1; } } println!("\n MEAN over {c} (bearing x elevation) cases: PathA {:.4}° PathB {:.4}° ratio {:.2}x", diff --git a/crates/helix/tests/signed360_claims.rs b/crates/helix/tests/signed360_claims.rs index d728fa75b..ef8466eaf 100644 --- a/crates/helix/tests/signed360_claims.rs +++ b/crates/helix/tests/signed360_claims.rs @@ -34,7 +34,10 @@ fn azimuth_spans_the_full_circle_not_merely_varies() { let covered = buckets.iter().filter(|b| **b).count(); assert_eq!(lo, 0, "azimuth must reach the bottom of the u16 range"); assert_eq!(hi, u16::MAX, "azimuth must reach the top of the u16 range"); - assert_eq!(covered, 256, "every 1/256 arc of the circle must be reachable"); + assert_eq!( + covered, 256, + "every 1/256 arc of the circle must be reachable" + ); } /// The two hemispheres occupy their EXACT halves of the `polar` byte. @@ -58,7 +61,11 @@ fn polar_partitions_are_exactly_the_two_halves() { neg_lo = neg_lo.min(q); neg_hi = neg_hi.max(q); } - assert_eq!((pos_lo, pos_hi), (128, 255), "Pos must fill [128,255] exactly"); + assert_eq!( + (pos_lo, pos_hi), + (128, 255), + "Pos must fill [128,255] exactly" + ); assert_eq!((neg_lo, neg_hi), (0, 127), "Neg must fill [0,127] exactly"); }