From 8788720877d85b389c4d53111b410aa46ea0ecb9 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 19:21:28 +0000 Subject: [PATCH 1/8] board: bank the Pillar-11 constant guard + the ndarray-substrate debt Two records, two lifecycles, one commit. E-PILLAR-11-PUBLISHED-BOUND-NEEDS-ITS-OWN-NUMERIC-GUARD-1 (EPIPHANIES) The arXiv-e -> Annals-2e correction doubles the L=3 certificate depth 7 -> 14 and the coefficient count 255 -> 32767. Measured: debug pillar_passes moves 25.98s -> 26.17s, i.e. a 128x coefficient increase costs ~0.2s inside a 26s run. The cost is the word enumeration (484 reduced words of length <= 3), not the signature depth. The 26s is the PDE depth-infinity leg, established by elimination and corroborated by the 0.18s release figure for a 4097-point solve -- not directly instrumented, and labelled as such. The load-bearing half: run with the WRONG constant, pillar_passes still passes; only theorem2_depth_is_the_paper_floor fails (L=1: 2 vs 4). That is correct division of labour -- the theorem bound is SUFFICIENT and the fixtures separate at depth 3, far inside either bound, so no behavioural test downstream of a sufficient bound can pin the constant. Consequence: theorem2_depth_is_the_paper_floor is an independent provenance guard and must not be deleted as redundant, however tautological it looks beside a green pillar. Generalised in the entry. TD-PILLAR11-SCIENTIFIC-LOOPS-BYPASS-NDARRAY-SIMD-1 (TECH_DEBT) Operator ruling: the debt is not missing SIMD support, it is scientific code bypassing the already-complete ndarray execution substrate. No violation was committed (zero raw intrinsics in sigker/jc), but the legs are hand-written scalar beside a substrate that owns the machine vocabulary. Corrected from an earlier weaker reading of mine: there is NO consumer-level "SIMD exists -> polyfill, else -> scalar fallback" decision, because scalar is a BACKEND of the same typed surface. Verified in simd.rs: F64x8 resolves to AVX-512 __m512d, AVX2 f64x4 x2, NEON [float64x2_t; 4], wasm32+simd128 [v128; 4], and scalar::F64x8 for other targets. The law is: call the named algorithm; if absent, COMPOSE from polyfill methods; never a consumer-local scalar arithmetic path, never consumer-local intrinsics. For the Goursat kernel the arithmetic is entirely available (mul_add, from_slice, copy_to_slice, reduce_sum all ship on F64x8). The only unsolved part is scheduling: k[i+1][j+1] reads k[i+1][j], a strict serial recurrence along j, so independence lives on the anti-diagonal. Preserved as the architectural reason not to jump from Vec to SIMD: A1 determines A2's shape. In a flat row-major buffer the anti-diagonal is strided (m-1) and needs gather; three rolling anti-diagonal buffers make every wavefront load contiguous and can eliminate the gather entirely. The storage decision fixes which lane ops A2 needs at all. W1.5 falsifier recorded, not run: A0 current Vec row-major reference; A1 flat/rolling storage, same recurrence and order, A0 = A1 EXACTLY; A2 rolling anti-diagonal via ndarray::simd::method(), A1 <-> A2 under a PREDECLARED tolerance because traversal changes evaluation order. Scalar inventory enumerated, split into canonical / accumulation-shaped / combinatorial. signature_pde_sweep and shuffle_product_lift are not shipped in ndarray -- the missing thing is traversal, not arithmetic. Pay by W1.5 (gated on jc Pillar 11, green for the lattice leg). NOT W5: W5 is workload machinery and stays HOLD at 4609 vs the 11585-point 1 GiB threshold regardless of any speedup. Supersession index regenerated after both board writes (per the regenerate-LAST rule); output byte-identical, so it is unchanged here. No code touched, W1.5 deliberately not implemented from this session. --- .claude/board/EPIPHANIES.md | 54 +++++++++++++++++++++++++ .claude/board/TECH_DEBT.md | 81 +++++++++++++++++++++++++++++++++++++ 2 files changed, 135 insertions(+) diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 853b1e8d7..8f8e8efdd 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,57 @@ +## 2026-09-02 — E-PILLAR-11-PUBLISHED-BOUND-NEEDS-ITS-OWN-NUMERIC-GUARD-1 + +**Status:** FINDING (measured) — closes the Hambly-Lyons edition-constant arc. +**Confidence:** every number below was run, not inferred; the disable arm was +executed and restored. + +The published constant correction (#1133/`1a3c7294`, arXiv `e` → +Annals `2e`) doubles the certificate depth. Measured consequences: + +| | old arXiv `e` | corrected Annals `2e` | +|---|---|---| +| `c` | `e·ln(1+√2)` = 2.3958 | `2e·ln(1+√2)` = 4.79155 | +| depth at `L=3` | 7 | **14** | +| coefficients (`d=2`) | 255 | **32 767** | +| `pillar_passes`, debug | 25.98 s | **26.17 s** | + +**A 128x coefficient increase costs ~0.2 s inside a 26 s run — inside noise.** +The exhaustive arm's cost is the enumeration of reduced words of length <= 3 +(484 of them), not the signature depth, so raising the cutoff moves nothing. +The other three tests are 0.07-0.10 s each, the `L=8` figure-of-eight arm +included: the correction only LOOSENED its bound (`<= 19` -> `<= 38`), making +it cheaper to satisfy. The 26 s is the PDE depth-infinity leg (8 Goursat solves +on 4609-point paths in debug) — established by elimination, since the constant +flip did not move it and the other tests are ~0.1 s, and corroborated by the +release figure of 0.18 s for a 4097-point solve. Not directly instrumented. + +**The load-bearing half: the behavioural tests do NOT guard the constant.** +Run with the wrong (arXiv `e`) constant, `pillar_passes` **still passes**. What +fails is `theorem2_depth_is_the_paper_floor`: + +``` +assertion `left == right` failed: L=1 + left: 2 floor(e·ln(1+√2)) + right: 4 floor(2e·ln(1+√2)) +``` + +That is the correct division of labour, not a gap. The theorem gives a +SUFFICIENT depth; the fixtures separate at depth 3, far inside either bound, so +a looser constant cannot fail a behaviour test that clears it comfortably. + +**Consequence, and the reason this is banked:** `theorem2_depth_is_the_paper_floor` +is an independent PROVENANCE guard — it is the only thing standing between the +tree and a silent regression to the arXiv edition. Beside a green +`pillar_passes` it reads like a tautology restating a constant, which is exactly +how a future tidy-up would justify deleting it. **It must not be removed as +redundant.** Numbering provenance is pinned at the definition site +(`hambly_lyons.rs:365`): Annals Theorem 5 = arXiv v2 Theorem 2; Annals +Theorem 2 is NOT the lattice cutoff. + +**Generalisation:** when a constant enters a SUFFICIENT bound, behavioural tests +downstream of it are structurally incapable of pinning it — any bound at least +as large passes. Such a constant needs its own numeric guard, and that guard's +apparent triviality is a property of correct design, not a smell. + ## 2026-09-02 — E-A-GHOST-TRACE-IS-NOT-THE-COUNTERFACTUAL-LANE-1 — the word "ghost" names two semantic families in the tree; the same-word-≠-same-family lesson of the September teardown, caught BEFORE a port this time **Status:** FINDING + FENCE (verified against source). **Confidence:** High. Raised by an external review of #1137 (GPT-class; held CLAIMED-BY-EXTERNAL per `E-EXTERNAL-REVIEW-ADJUDICATED-1` until checked, then confirmed on both sides in code). diff --git a/.claude/board/TECH_DEBT.md b/.claude/board/TECH_DEBT.md index ac81df884..5a3ed699a 100644 --- a/.claude/board/TECH_DEBT.md +++ b/.claude/board/TECH_DEBT.md @@ -1,3 +1,84 @@ +## TD-PILLAR11-SCIENTIFIC-LOOPS-BYPASS-NDARRAY-SIMD-1 (2026-09-02) — OPEN + +**The debt is not missing SIMD support. It is scientific code bypassing the +already-complete `ndarray` execution substrate.** (Operator ruling, 2026-09-02.) + +No violation was committed — the Pillar-11 legs contain zero raw intrinsics +(`core::arch` / `_mm*` / `target_feature`: 0 hits in `crates/sigker/src` and +`crates/jc/src`). They are scalar `f64`, hand-written beside a substrate that +already owns the machine vocabulary. + +**The law, corrected from an earlier weaker reading.** There is no +"SIMD op exists -> polyfill, else -> scalar fallback" decision at consumer +level. `ndarray::simd` hides the architecture choice behind ONE bit-exact typed +surface, and **scalar is one BACKEND of that surface**, not a consumer-authored +alternative: `F64x8` resolves to AVX-512 (`__m512d`), AVX2 (`f64x4` x2), NEON +(`[float64x2_t; 4]`), wasm32+simd128 (`[v128; 4]`), and `scalar::F64x8` for +"other non-x86 targets ... full scalar fallback" (`simd.rs` dispatch arms, +verified 2026-09-02). So: + +``` +named high-level algorithm exists -> call it +it does not exist -> COMPOSE it from ndarray polyfill methods +never -> a consumer-local scalar arithmetic path +never -> consumer-local intrinsics +``` + + JC tells us what is mathematically true. + ndarray tells the machine how to execute it. + Everything else is composition, never a second arithmetic implementation. + +**For the Goursat kernel the arithmetic is entirely available; the only +unsolved part is algorithmic SCHEDULING.** `signature_kernel_pde` +(`sigker/src/kernel.rs:106-131`) computes + +``` +k[i+1][j+1] = k[i+1][j] + k[i][j+1] - k[i][j] + c_ij·k[i][j] + = mul_add(c_ij, diag, left + up - diag) +``` + +`mul_add`, `from_slice`, `copy_to_slice`, `reduce_sum`, `select` all ship on +`F64x8` today. What blocks it is that `k[i+1][j+1]` reads `k[i+1][j]` — a +strict serial recurrence along `j` in row-major order. Independence lives on +the **anti-diagonal**; the transformation needed is +`row-major serial recurrence -> anti-diagonal / rolling-wavefront formulation`. + +**A1 determines A2's shape — the architectural reason not to jump from +`Vec` straight to SIMD.** In a flat row-major buffer the anti-diagonal is +STRIDED (stride `m-1`), so a naive wavefront needs gather. **Three rolling +anti-diagonal buffers make every wavefront load contiguous and can eliminate +the gather entirely.** The storage decision therefore fixes which lane ops A2 +needs at all; it is not a warm-up measurement. + +**W1.5 falsifier (do not run from a banking session):** + +| arm | shape | relation | +|---|---|---| +| A0 | current `Vec>`, row-major | reference | +| A1 | flat / rolling storage, SAME recurrence and order | **A0 = A1 exactly** — only storage changes | +| A2 | rolling anti-diagonal traversal via `ndarray::simd::method()` | A1 <-> A2 gets a PREDECLARED solver tolerance: traversal changes evaluation order | + +The A1<->A2 tolerance is stated in advance, never discovered after the fact. + +**Inventory — the scalar surfaces in the same two crates.** Straightforwardly +canonical (`reduce_sum` over `mul_add`): `LogSignature::{dot, cosine}`, +`RandomizedSignature::{dot, cosine}`, `linear_path_kernel_closed_form`. +Accumulation-shaped: `signature_truncated`, `log_signature_truncated`, +`RandomizedSignature::encode`, `hydrate_signature`, `signature_kernel`, +`signature_kernel_normalized`. Combinatorial rather than arithmetic, and +plausibly staying scalar: `shuffle_product`, `enumerate_lyndon_words`, +`witt_component` / `witt_dimension`. + +**Not shipped:** `signature_pde_sweep` and `shuffle_product_lift` return zero +hits in `ndarray/src/` — they remain the W1.5 catalogue's shopping list. The +missing thing is traversal/composition, NOT arithmetic; whether a composition +is later promoted to a named `signature_pde_sweep()` is purely an API/reuse +question, not a blocker. + +**Pay by:** W1.5, gated on `jc Pillar 11` (green for the lattice leg, +`7751581f`). **Not W5** — W5 is workload-pressure machinery and stays HOLD +(4609 vs the 11 585-point 1 GiB threshold) regardless of any speedup here. + ## TD-GHOST-TIER-NAME-COLLISION-1 (2026-09-02) — OPEN, doc-only `crates/lance-graph-contract/src/counterfactual.rs` calls the −6 minority-pole From d0f6960e2425d63118b89334de2b38c000947d90 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 19:24:51 +0000 Subject: [PATCH 2/8] board: pin the operator-confirmed ndarray backend parity in TD-PILLAR11 The dispatch check I banked established only that each backend EXPORTS F64x8. Operator confirmation, 2026-09-02: 'every backend implements every method with parity coverage.' That is the stronger fact a composing consumer actually depends on -- that a method reached through the typed surface is implemented, and bit-exact, on every arm -- and it is what makes 'compose from polyfill methods' a safe instruction rather than a per-method gamble. Without it the law still carried an implicit conditional; with it there is no consumer-side check left to make before A2. Appended in place inside the entry, dated and attributed. No existing sentence rewritten; the append-only board discipline is on the record, not on the sentence. Supersession index re-run after the write. --- .claude/board/TECH_DEBT.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.claude/board/TECH_DEBT.md b/.claude/board/TECH_DEBT.md index 5a3ed699a..9717efd0b 100644 --- a/.claude/board/TECH_DEBT.md +++ b/.claude/board/TECH_DEBT.md @@ -17,6 +17,14 @@ alternative: `F64x8` resolves to AVX-512 (`__m512d`), AVX2 (`f64x4` x2), NEON "other non-x86 targets ... full scalar fallback" (`simd.rs` dispatch arms, verified 2026-09-02). So: +**Method-level parity CONFIRMED (operator, 2026-09-02):** *"every backend +implements every method with parity coverage."* The dispatch check above only +established that each backend EXPORTS the type; this closes the stronger +question a composing consumer actually depends on — that a method reached +through the typed surface is implemented, and bit-exact, on every arm. It is +what makes "compose from polyfill methods" a safe instruction rather than a +per-method gamble, and it removes the last conditional from the law below. + ``` named high-level algorithm exists -> call it it does not exist -> COMPOSE it from ndarray polyfill methods From 5df2d785d5ad15acdf618686a5dd63b0716eabfb Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 19:47:10 +0000 Subject: [PATCH 3/8] =?UTF-8?q?jc:=20the=20Goursat=20solve=20on=20the=20nd?= =?UTF-8?q?array=20substrate=20=E2=80=94=20A0/A1/A2=20measured;=20ndarray?= =?UTF-8?q?=20mandatory;=20v3/v4/native=20configs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Operator: "you didn't try the 25-26 seconds with ndarray yet" — then "so why don't you use mul_add". Done. crates/jc/examples/goursat_substrate_probe.rs runs the falsifier banked in TD-PILLAR11-SCIENTIFIC-LOOPS-BYPASS-NDARRAY-SIMD-1 with every A2 lane op = ndarray::simd::F64x8::mul_add. The wavefront body is three FMAs (t = mul_add(1,left,up); u = mul_add(-1,diag,t); new = mul_add(c,diag,u)). No Add/Sub operator needed, none minted: I had been one step from filing the F64x8 operator-trait asymmetry as a parity defect when the composition was available all along. Release, 4097-point paths, 16.8M cells: target-cpu A0(Vec) A1(flat) A2(F64x8) A1/A2 A0=A1 generic x86-64 ("386") 0.182 s 0.169 s 0.773 s 0.22x exact x86-64-v3 (AVX2) 0.163 s 0.150 s 0.0165 s 9.12x exact x86-64-v4 (AVX-512) 0.171 s 0.157 s 0.0182 s 8.62x exact Four findings, each measured: 1. Storage was not the wall; the recurrence was. A1/A0 = 1.09x. My banked hypothesis that flat storage alone would close most of the gap is FALSIFIED — the probe was built to answer that and it did. 2. The "386" build is a regression, not a no-op: at generic x86-64 the polyfill's scalar arm runs A2 4.5x SLOWER than the scalar loop. Until this commit lance-graph had no .cargo/config.toml, so every local build landed there. CI was already v3 via RUSTFLAGS in .github/workflows/*.yml. 3. Bit-exact across backends, as the operator confirmed: |A1-A2|/A1 is identical to the last digit on scalar, AVX2 and AVX-512 at every size. The ~1e-13 delta is fused-vs-separate rounding of c·diag over 16.8M cells; A2 is the MORE accurate arm. 4. AVX2 ~ AVX-512 (9.1x vs 8.6x): latency-bound on the diagonal recurrence, not width-bound. Widening lanes buys nothing until the dependency chain is restructured — a scheduling question, not a substrate one. Storage detail that held: dy stored reversed makes the anti-diagonal walk forward in i, so k-buffers, dx and dy are all contiguous slices. No gather. Configs (operator: "GitHub needs v3, everything else v4 or native"): .cargo/config.toml x86-64-v3 default, mirrors ndarray's rationale (below v3 SIGILLs on simd_avx2 intrinsics) .cargo/config-avx512.toml sapphirerapids, mirrors ndarray's .cargo/config-native.toml native, mirrors ndarray's ndarray's two crypto cfgs (curve25519_dalek_backend, poly1305_force_soft) are deliberately NOT repeated: neither crate is in this workspace's Cargo.lock (verified, 0 hits). Note .cargo/config.toml is read from the INVOKING directory, so ndarray's own config never applied to builds from here. jc: ndarray is now a plain, non-optional [dependencies] entry (operator: mandatory everywhere). The "zero external deps in production" header is retired and rewritten; what stays standalone is the PROOF. The earlier dev-dep placement is removed. Verified release: 127 + 13 tests pass with --features hambly-lyons; default-feature build (no sigker) clean. Board: TD entry extended with the results table and the four findings. Supersession index re-run after the board write; unchanged. Not done, deliberately: no change to sigker's shipped signature_kernel_pde. The probe measures; W1.5 decides whether the wavefront replaces the solver and whether the composition is promoted to a named signature_pde_sweep. No debug builds (operator: 24 min / 18 GB — release is 6 min / 3 GB). Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp --- .cargo/config-avx512.toml | 7 + .cargo/config-native.toml | 6 + .cargo/config.toml | 34 ++++ .claude/board/TECH_DEBT.md | 42 +++++ crates/jc/Cargo.lock | 87 ++++++++++ crates/jc/Cargo.toml | 27 +-- crates/jc/examples/goursat_substrate_probe.rs | 161 ++++++++++++++++++ 7 files changed, 353 insertions(+), 11 deletions(-) create mode 100644 .cargo/config-avx512.toml create mode 100644 .cargo/config-native.toml create mode 100644 .cargo/config.toml create mode 100644 crates/jc/examples/goursat_substrate_probe.rs diff --git a/.cargo/config-avx512.toml b/.cargo/config-avx512.toml new file mode 100644 index 000000000..3a5543578 --- /dev/null +++ b/.cargo/config-avx512.toml @@ -0,0 +1,7 @@ +# AVX-512 build. Mirrors ndarray/.cargo/config-avx512.toml (`sapphirerapids`, +# which implies x86-64-v4 plus AMX/BF16/VNNI). Use when the deployment silicon +# is known: cargo build --config .cargo/config-avx512.toml ... +# The `simd.rs` dispatch picks the `simd_avx512` backend (native __m512d). +[build] +[target.'cfg(target_arch = "x86_64")'] +rustflags = ["-Ctarget-cpu=sapphirerapids"] diff --git a/.cargo/config-native.toml b/.cargo/config-native.toml new file mode 100644 index 000000000..bf9ed2396 --- /dev/null +++ b/.cargo/config-native.toml @@ -0,0 +1,6 @@ +# Build-machine-tuned. rustc resolves the host CPUID at compile time; the +# binary is NOT portable off this machine. Mirrors ndarray/.cargo/config-native.toml. +# cargo build --config .cargo/config-native.toml ... +[build] +[target.'cfg(target_arch = "x86_64")'] +rustflags = ["-Ctarget-cpu=native"] diff --git a/.cargo/config.toml b/.cargo/config.toml new file mode 100644 index 000000000..bfcfa57b5 --- /dev/null +++ b/.cargo/config.toml @@ -0,0 +1,34 @@ +# lance-graph cargo config — mirrors ndarray's `.cargo/config.toml` policy +# (operator ruling, 2026-09-02): GitHub needs v3; everything else v4 or native. +# +# ndarray is the MANDATORY execution substrate for every crate here, and its +# `simd.rs` dispatch is COMPILE-TIME (`cfg(target_feature = "avx512f")` / +# `"avx2"`). Whatever `target-cpu` this build runs at decides which backend +# `ndarray::simd::{F64x8, F32x16, I16x32}` resolves to. `.cargo/config.toml` +# is read from the INVOKING directory, not the dependency's — so ndarray's +# own config does NOT apply when a lance-graph crate is built from here. +# Without this file a local build compiles the substrate at generic x86-64 +# (SSE2), i.e. the scalar arm. +# +# Measured on the Goursat wavefront (crates/jc/examples/goursat_substrate_probe, +# 4097-point paths, release): under v4 `F64x8::mul_add` is 8.6x faster than +# the flat scalar loop; at generic x86-64 the SAME code is 4.5x SLOWER than +# scalar, because the polyfill's scalar arm is emulating eight lanes through +# arrays. The "386" build is a regression, not a no-op. +# +# Default here is v3 (AVX2) for the same reason ndarray chose it: portable +# across all x86_64 since Haswell, and what CI runs (`.github/workflows/*.yml` +# set `-C target-cpu=x86-64-v3` explicitly). Anything below v3 SIGILLs on the +# `simd_avx2` intrinsics. +# +# AVX-512 (v4): cargo build --config .cargo/config-avx512.toml ... +# or CARGO_BUILD_RUSTFLAGS='-Ctarget-cpu=x86-64-v4' +# Build machine: cargo build --config .cargo/config-native.toml ... +# +# The two crypto cfgs ndarray's config also carries (`curve25519_dalek_backend`, +# `poly1305_force_soft`) are deliberately NOT repeated: neither crate is in +# this workspace's Cargo.lock (verified 2026-09-02, 0 hits), so there is no +# AVX2 crypto surface here to compile out. Re-check if `encryption` or +# `ed25519-dalek` ever enters the tree. +[target.'cfg(target_arch = "x86_64")'] +rustflags = ["-Ctarget-cpu=x86-64-v3"] diff --git a/.claude/board/TECH_DEBT.md b/.claude/board/TECH_DEBT.md index 9717efd0b..01193f6a4 100644 --- a/.claude/board/TECH_DEBT.md +++ b/.claude/board/TECH_DEBT.md @@ -87,6 +87,48 @@ question, not a blocker. `7751581f`). **Not W5** — W5 is workload-pressure machinery and stays HOLD (4609 vs the 11 585-point 1 GiB threshold) regardless of any speedup here. +**A0/A1/A2 RUN (2026-09-02, same day — operator: "you didn't try the 25-26 +seconds with ndarray yet").** `crates/jc/examples/goursat_substrate_probe.rs`, +release, 4097-point paths (16.8M cells). Every A2 lane op is +`ndarray::simd::F64x8::mul_add` — the body is three FMAs +(`t = mul_add(1,left,up)`, `u = mul_add(-1,diag,t)`, `new = mul_add(c,diag,u)`); +no `Add`/`Sub` operator was needed, and none was minted. + +| target-cpu | backend | A0 | A1 | A2 | A1/A2 | A0=A1 | \|A1-A2\|/A1 | +|---|---|---|---|---|---|---|---| +| generic x86-64 ("386") | scalar arm | 0.182 s | 0.169 s | **0.773 s** | **0.22x** | bit-exact | 2.431e-13 | +| x86-64-v3 (AVX2) | `f64x4` x2 | 0.163 s | 0.150 s | 0.0165 s | **9.12x** | bit-exact | 2.431e-13 | +| x86-64-v4 (AVX-512) | `__m512d` | 0.171 s | 0.157 s | 0.0182 s | **8.62x** | bit-exact | 2.431e-13 | + +Four findings, each falsifiable and each measured: + +1. **Storage was NOT the wall; the recurrence was.** A1/A0 = 1.09x. The + hypothesis banked above — that a flat buffer alone would close most of the + gap — is **FALSIFIED**. The probe was built to answer that and it did. +2. **The "386" build is a REGRESSION, not a no-op.** At generic x86-64 the + polyfill's scalar arm runs the wavefront 4.5x SLOWER than the flat scalar + loop: eight lanes emulated through arrays plus the wavefront bookkeeping. + Until this commit lance-graph had NO `.cargo/config.toml`, so every local + build landed there. `.cargo/config.toml` now pins v3 (CI already did via + `RUSTFLAGS` in `.github/workflows/*.yml`), with `config-avx512.toml` / + `config-native.toml` mirroring ndarray's. +3. **Bit-exact across backends, as confirmed.** `|A1-A2|/A1` is identical to the + last digit on scalar, AVX2 and AVX-512 at every size. The delta itself + (~1e-13 at 4097) is the fused-vs-separate rounding of `c·diag` + accumulated over 16.8M cells — A2 is the MORE accurate arm. +4. **AVX2 ~ AVX-512 here** (9.1x vs 8.6x): the wavefront is latency-bound on + the diagonal recurrence, not width-bound. Widening lanes buys nothing until + the dependency chain is restructured; that is a scheduling question, not a + substrate one. + +Storage detail that held: with `dy` stored REVERSED, the anti-diagonal walk +is forward in `i`, so k-buffers, `dx` and `dy` are all contiguous slices — +**no gather**, exactly as predicted by "A1 determines A2's shape". + +`jc` now depends on `ndarray` as a plain, non-optional `[dependencies]` +entry; its "zero external deps in production" header is retired (operator: +ndarray is mandatory everywhere). What stays standalone is the PROOF. + ## TD-GHOST-TIER-NAME-COLLISION-1 (2026-09-02) — OPEN, doc-only `crates/lance-graph-contract/src/counterfactual.rs` calls the −6 minority-pole diff --git a/crates/jc/Cargo.lock b/crates/jc/Cargo.lock index 9be3afa8b..550bd302f 100644 --- a/crates/jc/Cargo.lock +++ b/crates/jc/Cargo.lock @@ -11,6 +11,12 @@ dependencies = [ "memchr", ] +[[package]] +name = "autocfg" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" + [[package]] name = "equivalent" version = "1.0.2" @@ -57,6 +63,7 @@ name = "jc" version = "0.1.0" dependencies = [ "lance-graph-contract", + "ndarray", "neural-debug", "p64", "sigker", @@ -77,12 +84,38 @@ version = "0.2.16" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b6d2cec3eae94f9f509c767b45932f1ada8350c4bdb85af2fcab4a3c14807981" +[[package]] +name = "matrixmultiply" +version = "0.3.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f607c237553f086e7043417a51df26b2eb899d3caff94e6a67592ff992fedc7" +dependencies = [ + "autocfg", + "rawpointer", +] + [[package]] name = "memchr" version = "2.8.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" +[[package]] +name = "ndarray" +version = "0.17.2" +dependencies = [ + "fractal", + "matrixmultiply", + "num-complex", + "num-integer", + "num-traits", + "p64", + "paste", + "portable-atomic", + "portable-atomic-util", + "rawpointer", +] + [[package]] name = "neural-debug" version = "0.1.0" @@ -93,6 +126,33 @@ dependencies = [ "walkdir", ] +[[package]] +name = "num-complex" +version = "0.4.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495" +dependencies = [ + "num-traits", +] + +[[package]] +name = "num-integer" +version = "0.1.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7ce2d95d4b3734dc35aa2f45e1aa22cd416814592a4f9d9205e11affd5b8e10b" +dependencies = [ + "num-traits", +] + +[[package]] +name = "num-traits" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" +dependencies = [ + "autocfg", +] + [[package]] name = "p64" version = "0.1.0" @@ -100,6 +160,27 @@ dependencies = [ "fractal", ] +[[package]] +name = "paste" +version = "1.0.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a" + +[[package]] +name = "portable-atomic" +version = "1.15.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "05c8b63e8d9609db387f0324918f81d68fe27748f084ef092fb35954d0539a85" + +[[package]] +name = "portable-atomic-util" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2a106d1259c23fac8e543272398ae0e3c0b8d33c88ed73d0cc71b0f1d902618" +dependencies = [ + "portable-atomic", +] + [[package]] name = "proc-macro2" version = "1.0.106" @@ -118,6 +199,12 @@ dependencies = [ "proc-macro2", ] +[[package]] +name = "rawpointer" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "60a357793950651c4ed0f3f52338f53b2f809f32d83a07f72909fa13e4c6c1e3" + [[package]] name = "regex" version = "1.13.1" diff --git a/crates/jc/Cargo.toml b/crates/jc/Cargo.toml index 639f3a9d4..2a5bab21e 100644 --- a/crates/jc/Cargo.toml +++ b/crates/jc/Cargo.toml @@ -5,31 +5,36 @@ edition = "2021" description = "Jirak-Cartan: five-pillar proof-in-code for binary-Hamming causal field computation" license = "Apache-2.0" -# Zero EXTERNAL deps in production — standalone, like deepnsm and bgz17. -# The proof is the proof regardless of SIMD path. -# Workspace-sibling path-deps are opt-in via features (see [features] below). - -# Optional workspace-sibling deps — gated by feature flags to preserve -# the default zero-dep build. `cargo build` (default) gives a fully -# standalone JC; `cargo build --features hambly-lyons` activates Pillar 11 -# by pulling in the sigker workspace sibling. +# ndarray is MANDATORY — here and everywhere in lance-graph (operator ruling, +# 2026-09-02). It is the execution substrate: every numeric path is written +# against `ndarray::simd::{F64x8, F32x16, I16x32}::*`, whose scalar / AVX2 / +# AVX-512 / NEON / WASM backends are bit-exact behind one typed surface. JC +# certifies what is mathematically true; ndarray tells the machine how to +# execute it. A consumer-local scalar arithmetic path is not a fallback, it +# is a second arithmetic implementation, and is forbidden. +# +# The former "zero external deps in production" constitution is retired by +# that ruling. What remains standalone is the PROOF: no other crate's +# behaviour is a premise of any pillar. Workspace siblings that carry +# domain content (sigker) stay feature-gated so a pillar can be run alone. [dependencies] +ndarray = { path = "../../../ndarray" } sigker = { path = "../sigker", optional = true } [features] -# Default build is zero-dep — honors the standalone constitution. +# Default build: ndarray only (mandatory). Pillar 11 opts in below. default = [] # Activates Pillar 11 (Hambly-Lyons signature uniqueness) by pulling in the # sigker workspace sibling. See `src/hambly_lyons.rs` for the probe + the # DEFERRED fallback used when the feature is off. hambly-lyons = ["dep:sigker"] -# Dev-only deps for cross-crate bridge examples (production stays zero-dep). +# Dev-only deps for cross-crate bridge examples. [dev-dependencies] lance-graph-contract = { path = "../lance-graph-contract" } # `substrate_compare` example only — the on-demand P64-vs-V3 comparison drives # the REAL Palette64 (compute) + NodeGuid (address) and records into the -# neural-debug runtime registry. Dev-deps, so production JC stays zero-dep. +# neural-debug runtime registry. Dev-deps: examples only, not the library. p64 = { path = "../../../ndarray/crates/p64" } neural-debug = { path = "../neural-debug" } diff --git a/crates/jc/examples/goursat_substrate_probe.rs b/crates/jc/examples/goursat_substrate_probe.rs new file mode 100644 index 000000000..6d385237f --- /dev/null +++ b/crates/jc/examples/goursat_substrate_probe.rs @@ -0,0 +1,161 @@ +//! A0/A1/A2 — the Goursat solve on the canonical ndarray substrate. +//! +//! The falsifier recorded in `TD-PILLAR11-SCIENTIFIC-LOOPS-BYPASS-NDARRAY-SIMD-1`: +//! +//! | arm | storage | traversal | arithmetic | +//! |---|---|---|---| +//! | A0 | `Vec>` | row-major | scalar (the shipped `signature_kernel_pde`) | +//! | A1 | flat `Vec` | row-major, SAME order | scalar | +//! | A2 | three rolling anti-diagonals | wavefront | `ndarray::simd::F64x8::mul_add` | +//! +//! Contract: A0 = A1 **bit-exact** (only storage changes). A1 <-> A2 under a +//! predeclared tolerance: the wavefront's `mul_add(c, diag, u)` fuses the +//! `c·diag + u` rounding the scalar path performs as two roundings, so the two +//! arms legitimately differ in the last bits. No consumer intrinsics; every +//! lane op is `ndarray::simd::method()`. +//! +//! Run: `cargo run --release --manifest-path crates/jc/Cargo.toml \ +//! --features hambly-lyons --example goursat_substrate_probe` + +use ndarray::simd::F64x8; +use sigker::signature_kernel_pde; +use std::time::Instant; + +const LANES: usize = 8; + +fn path(n: usize) -> Vec> { + (0..=n) + .map(|i| { + let t = i as f64 / n as f64; + vec![t + 0.05 * (260.0 * t).cos(), 0.5 * t + 0.05 * (260.0 * t).sin()] + }) + .collect() +} + +/// Increments dx_i = x[i+1] - x[i], one contiguous block per step. +fn increments(x: &[Vec]) -> (Vec, usize) { + let dim = x[0].len(); + let mut out = Vec::with_capacity((x.len() - 1) * dim); + for w in x.windows(2) { + for a in 0..dim { + out.push(w[1][a] - w[0][a]); + } + } + (out, dim) +} + +/// A1 — flat row-major storage, the shipped recurrence in the shipped order. +/// `k[i+1][j+1] = k[i+1][j] + k[i][j+1] - k[i][j] + c_ij * k[i][j]`, evaluated +/// left-to-right exactly as `signature_kernel_pde` writes it. +fn goursat_flat(x: &[Vec], y: &[Vec]) -> f64 { + let (n, m) = (x.len(), y.len()); + let (dx, dim) = increments(x); + let (dy, _) = increments(y); + let mut k = vec![1.0f64; n * m]; + for i in 0..n - 1 { + let dxi = &dx[i * dim..(i + 1) * dim]; + for j in 0..m - 1 { + let dyj = &dy[j * dim..(j + 1) * dim]; + let c: f64 = (0..dim).map(|a| dxi[a] * dyj[a]).sum(); + let (left, up, diag) = (k[(i + 1) * m + j], k[i * m + j + 1], k[i * m + j]); + k[(i + 1) * m + j + 1] = left + up - diag + c * diag; + } + } + k[n * m - 1] +} + +/// A2 — wavefront over three rolling anti-diagonal buffers, every lane op +/// `ndarray::simd::F64x8::mul_add`. +/// +/// On diagonal `d` (cells with `i + j = d`), indexed by row `i`: +/// left = k[i][j-1] = prev1[i] up = k[i-1][j] = prev1[i-1] +/// diag = k[i-1][j-1] = prev2[i-1] c = +/// With `dy` stored REVERSED, `dy[j-1] = dyr[m-1-d+i]` walks FORWARD in `i`, +/// so every operand — k-buffers, dx, dy — is a contiguous slice. No gather. +/// +/// Body, three FMAs (±1.0 multipliers are exact, so the first two round +/// exactly like `+`/`-`; only the last fuses what A1 rounds twice): +/// t = mul_add( 1, left, up) u = mul_add(-1, diag, t) new = mul_add(c, diag, u) +fn goursat_wavefront(x: &[Vec], y: &[Vec]) -> f64 { + let (n, m) = (x.len(), y.len()); + let (dx, dim) = increments(x); + let (dy, _) = increments(y); + assert_eq!(dim, 2, "probe fixes dim = 2 (the W5 path shape)"); + // Split by component; reverse dy so the anti-diagonal walk is forward. + let dx0: Vec = dx.iter().step_by(2).copied().collect(); + let dx1: Vec = dx.iter().skip(1).step_by(2).copied().collect(); + let mut dyr0: Vec = dy.iter().step_by(2).copied().collect(); + let mut dyr1: Vec = dy.iter().skip(1).step_by(2).copied().collect(); + dyr0.reverse(); + dyr1.reverse(); + + let mut prev2 = vec![1.0f64; n]; + let mut prev1 = vec![1.0f64; n]; + let mut cur = vec![1.0f64; n]; + let (one, neg_one, zero) = (F64x8::splat(1.0), F64x8::splat(-1.0), F64x8::splat(0.0)); + let mut out = [0.0f64; LANES]; + + for d in 2..(n + m - 1) { + // Boundaries on this diagonal: k[0][d] and k[d][0] are 1. + if d < m { cur[0] = 1.0; } + if d < n { cur[d] = 1.0; } + // Interior rows: i >= 1, j = d - i >= 1, i <= n-1, j <= m-1. + let lo = 1usize.max(d.saturating_sub(m - 1)); + let hi = (d - 1).min(n - 1); + if lo > hi { std::mem::swap(&mut prev2, &mut prev1); std::mem::swap(&mut prev1, &mut cur); continue; } + // dyr index for row i is (m-1-d)+i; the difference may be negative but the sum is not. + let base = (m - 1).wrapping_sub(d); + let mut i = lo; + while i + LANES <= hi + 1 { + let left = F64x8::from_slice(&prev1[i..i + LANES]); + let up = F64x8::from_slice(&prev1[i - 1..i - 1 + LANES]); + let diag = F64x8::from_slice(&prev2[i - 1..i - 1 + LANES]); + let a0 = F64x8::from_slice(&dx0[i - 1..i - 1 + LANES]); + let a1 = F64x8::from_slice(&dx1[i - 1..i - 1 + LANES]); + let r = base.wrapping_add(i); // == m-1-d+i, in range for interior rows + let b0 = F64x8::from_slice(&dyr0[r..r + LANES]); + let b1 = F64x8::from_slice(&dyr1[r..r + LANES]); + let c = a1.mul_add(b1, a0.mul_add(b0, zero)); + let t = one.mul_add(left, up); + let u = neg_one.mul_add(diag, t); + c.mul_add(diag, u).copy_to_slice(&mut out); + cur[i..i + LANES].copy_from_slice(&out); + i += LANES; + } + // Scalar tail: same three-FMA arithmetic, so A2 is internally uniform. + while i <= hi { + let r = base.wrapping_add(i); + let c = dx1[i - 1].mul_add(dyr1[r], dx0[i - 1] * dyr0[r]); + let t = 1.0f64.mul_add(prev1[i], prev1[i - 1]); + let u = (-1.0f64).mul_add(prev2[i - 1], t); + cur[i] = c.mul_add(prev2[i - 1], u); + i += 1; + } + std::mem::swap(&mut prev2, &mut prev1); + std::mem::swap(&mut prev1, &mut cur); + } + // Final cell k[n-1][m-1] is on diagonal n+m-2, the last computed, now in prev1. + prev1[n - 1] +} + +fn main() { + println!( + "{:>6} {:>10} {:>10} {:>10} {:>9} {:>9} {:>12} {:>12}", + "len", "A0 secs", "A1 secs", "A2 secs", "A0/A1", "A1/A2", "A0==A1", "|A1-A2|/A1" + ); + for &n in &[256usize, 1024, 2048, 4096] { + let (x, y) = (path(n), path(n)); + let t = Instant::now(); let a0 = signature_kernel_pde(&x, &y); let s0 = t.elapsed().as_secs_f64(); + let t = Instant::now(); let a1 = goursat_flat(&x, &y); let s1 = t.elapsed().as_secs_f64(); + let t = Instant::now(); let a2 = goursat_wavefront(&x, &y); let s2 = t.elapsed().as_secs_f64(); + let exact = a0.to_bits() == a1.to_bits(); + let rel = ((a1 - a2) / a1).abs(); + println!( + "{:>6} {s0:>10.4} {s1:>10.4} {s2:>10.4} {:>8.2}x {:>8.2}x {:>12} {rel:>12.3e}", + n + 1, s0 / s1, s1 / s2, if exact { "bit-exact" } else { "DIFFERS" } + ); + assert!(exact, "A0 != A1 at n={n}: {a0:e} vs {a1:e} — storage change altered the result"); + } + println!("\nA0 = A1 bit-exact at every size (only storage changed)."); + println!("A1 <-> A2 differ by fused vs. separate rounding of c·diag — the predeclared tolerance."); +} From e87af51e9951eac75e5dc356545ca6b989053ac4 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 20:06:55 +0000 Subject: [PATCH 4/8] config + board: drop the "386" label I misread into a terse message; note sapphirerapids is a superset of Cascade/Ice Lake MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Operator wrote "v4 386" meaning x86-64-v4. I read "386" as a warning about a baseline-i386 build and wrote that label into .cargo/config.toml, the TD entry, the commit message and the PR body — attributing the phrase to the operator. The measurement it labelled is unchanged and stands: a build with NO target-cpu runs the polyfill's scalar arm and is 4.5x slower than the scalar loop on the Goursat wavefront. Only the label was wrong. Fixed here: .cargo/config.toml comment; the TD table row and finding-2 heading; a dated correction line appended inside the TD entry (append-only — nothing else in the entry rewritten). The commit message on 5df2d785 keeps the label; it is pushed and is not rewritten. Second fix, same PR: .cargo/config-avx512.toml mirrors ndarray's `sapphirerapids`, which is a SUPERSET of most AVX-512 silicon — it implies x86-64-v4 PLUS avx512_bf16 / avx512_fp16 / AMX. Measured this session: this host is Cascade Lake (family 6, model 0x55, avx512_vnni, no amx_tile; amx_report(): cpu_model()=OtherX86, expects_amx=false, available=false). AMX stays inert there (inline-asm, runtime-gated), but anything compiled under cfg(target_feature = "avx512bf16") would SIGILL. The config now says so and names the alternatives: x86-64-v4 (what the probe was actually run with, so its numbers are unaffected) or native. Not adjudicated: the operator's statement that Sapphire and Emerald Rapids are ENABLED differently. ndarray's enablement doc §2 says the opposite (the arch_prctl grant is silicon-independent; EMR was merely the first host where gate 4 was fixed). Untestable on Cascade Lake; needs amx_report() on each. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp --- .cargo/config-avx512.toml | 17 ++++++++++++++--- .cargo/config.toml | 2 +- .claude/board/TECH_DEBT.md | 15 +++++++++++++-- 3 files changed, 28 insertions(+), 6 deletions(-) diff --git a/.cargo/config-avx512.toml b/.cargo/config-avx512.toml index 3a5543578..721e59fbb 100644 --- a/.cargo/config-avx512.toml +++ b/.cargo/config-avx512.toml @@ -1,7 +1,18 @@ -# AVX-512 build. Mirrors ndarray/.cargo/config-avx512.toml (`sapphirerapids`, -# which implies x86-64-v4 plus AMX/BF16/VNNI). Use when the deployment silicon -# is known: cargo build --config .cargo/config-avx512.toml ... +# AVX-512 build. Mirrors ndarray/.cargo/config-avx512.toml (`sapphirerapids`). +# cargo build --config .cargo/config-avx512.toml ... # The `simd.rs` dispatch picks the `simd_avx512` backend (native __m512d). +# +# `sapphirerapids` is a SUPERSET of most AVX-512 silicon in the fleet. It +# implies x86-64-v4 PLUS avx512_bf16 / avx512_fp16 / AMX. On Sapphire / +# Emerald / Granite Rapids that is the right pick. On Cascade Lake (family 6 +# model 0x55 — what this session's host is) or Ice Lake, it is NOT: AMX is +# inline-asm and runtime-gated by `amx_available()`, so it stays inert, but +# anything compiled under `cfg(target_feature = "avx512bf16")` will SIGILL on +# silicon that lacks it. There, use one of: +# CARGO_BUILD_RUSTFLAGS='-Ctarget-cpu=x86-64-v4' # what the probe was run with +# cargo build --config .cargo/config-native.toml # host CPUID, non-portable +# Rule of thumb: `sapphirerapids` for a KNOWN SPR/EMR/GNR deployment target; +# `x86-64-v4` for portable AVX-512; `native` for this-machine-only. [build] [target.'cfg(target_arch = "x86_64")'] rustflags = ["-Ctarget-cpu=sapphirerapids"] diff --git a/.cargo/config.toml b/.cargo/config.toml index bfcfa57b5..6dd5d98a7 100644 --- a/.cargo/config.toml +++ b/.cargo/config.toml @@ -14,7 +14,7 @@ # 4097-point paths, release): under v4 `F64x8::mul_add` is 8.6x faster than # the flat scalar loop; at generic x86-64 the SAME code is 4.5x SLOWER than # scalar, because the polyfill's scalar arm is emulating eight lanes through -# arrays. The "386" build is a regression, not a no-op. +# arrays. A build with no target-cpu is a regression, not a no-op. # # Default here is v3 (AVX2) for the same reason ndarray chose it: portable # across all x86_64 since Haswell, and what CI runs (`.github/workflows/*.yml` diff --git a/.claude/board/TECH_DEBT.md b/.claude/board/TECH_DEBT.md index 01193f6a4..8cc691c39 100644 --- a/.claude/board/TECH_DEBT.md +++ b/.claude/board/TECH_DEBT.md @@ -96,7 +96,7 @@ no `Add`/`Sub` operator was needed, and none was minted. | target-cpu | backend | A0 | A1 | A2 | A1/A2 | A0=A1 | \|A1-A2\|/A1 | |---|---|---|---|---|---|---|---| -| generic x86-64 ("386") | scalar arm | 0.182 s | 0.169 s | **0.773 s** | **0.22x** | bit-exact | 2.431e-13 | +| generic x86-64 (no target-cpu) | scalar arm | 0.182 s | 0.169 s | **0.773 s** | **0.22x** | bit-exact | 2.431e-13 | | x86-64-v3 (AVX2) | `f64x4` x2 | 0.163 s | 0.150 s | 0.0165 s | **9.12x** | bit-exact | 2.431e-13 | | x86-64-v4 (AVX-512) | `__m512d` | 0.171 s | 0.157 s | 0.0182 s | **8.62x** | bit-exact | 2.431e-13 | @@ -105,7 +105,7 @@ Four findings, each falsifiable and each measured: 1. **Storage was NOT the wall; the recurrence was.** A1/A0 = 1.09x. The hypothesis banked above — that a flat buffer alone would close most of the gap — is **FALSIFIED**. The probe was built to answer that and it did. -2. **The "386" build is a REGRESSION, not a no-op.** At generic x86-64 the +2. **A build with no `target-cpu` is a REGRESSION, not a no-op.** At generic x86-64 the polyfill's scalar arm runs the wavefront 4.5x SLOWER than the flat scalar loop: eight lanes emulated through arrays plus the wavefront bookkeeping. Until this commit lance-graph had NO `.cargo/config.toml`, so every local @@ -121,6 +121,17 @@ Four findings, each falsifiable and each measured: the dependency chain is restructured; that is a scheduling question, not a substrate one. +**Correction (2026-09-02, same session):** an earlier revision of this entry +labelled the no-`target-cpu` row `"386"` and attributed the phrase to the +operator. That was a misreading of `x86-64-v4` in a terse message. The +measurement is unchanged; only the label was wrong and is removed above. The +commit message on `5df2d785` still carries it and is not rewritten (pushed). +Also recorded: `.cargo/config-avx512.toml`'s `sapphirerapids` is a SUPERSET of +Cascade Lake / Ice Lake silicon — this session's host is Cascade Lake (family 6 +model 0x55; `amx_report()`: `cpu_model()=OtherX86`, `expects_amx=false`). On +such hosts `x86-64-v4` or `native` is the correct pick; the probe was run with +`x86-64-v4`, so its numbers stand. + Storage detail that held: with `dy` stored REVERSED, the anti-diagonal walk is forward in `i`, so k-buffers, `dx` and `dy` are all contiguous slices — **no gather**, exactly as predicted by "A1 determines A2's shape". From 3f347badbeb8b3504ce100277d83f8ad3fc4863c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 20:12:41 +0000 Subject: [PATCH 5/8] =?UTF-8?q?board:=20SPR=20vs=20EMR=20resolved=20?= =?UTF-8?q?=E2=80=94=20the=20detection=20changed,=20not=20the=20enablement?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Operator refinement: "maybe the enablement isn't different but the CPU detection was." Checked against ndarray git history, and that is exactly it. Pre-PR-#217 (src/simd_caps.rs @ bdf243cc, 2026-06-13): AMX detected by CPUID feature bits alone — amx_tile / amx_int8 / amx_bf16 / amx_fp16, EDX bits 24/25/22. No XCR0 gate, no model table, arch_prctl on syscall 157 (always -EINVAL). On Sapphire Rapids that detector said "present" while no tile op ever executed: every AMX test early-returned, every checkmark was aspiration (Gotcha 9). PR #217 (e563fdcd, 2026-06-14): the four-gate detector (CPUID + OSXSAVE + XCR0 + arch_prctl on 158) PLUS the CPUID model table — CpuModel with SPR 0x8F, EMR 0xCF, GNR 0xAD/0xAE, SRF 0xAF — added to distinguish "no silicon" from "present but not OS-enabled". On Emerald Rapids that detector said "present and enabled" and tiles ran, bit-exact. So: enablement identical on both (the arch_prctl grant is silicon-independent, as amx-enablement-and-kernel.md §2 says); DETECTION changed between the two eras, and the change happened to land on EMR silicon. From the outside that reads as "SPR and EMR detected differently". Both descriptions are true; the doc's §2 sentence "EMR was simply the host where gate 4 got fixed first" is the same fact stated from inside the code. Replaces the "not adjudicated" line in the TD entry with the resolution, dated. One minor inconsistency noticed and left alone: cpu_ops.rs:186 says "Linux 5.19+" for the arch_prctl requirement, the knowledge doc says "5.16+". Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp --- .claude/board/TECH_DEBT.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/.claude/board/TECH_DEBT.md b/.claude/board/TECH_DEBT.md index 8cc691c39..91e87705a 100644 --- a/.claude/board/TECH_DEBT.md +++ b/.claude/board/TECH_DEBT.md @@ -132,6 +132,23 @@ model 0x55; `amx_report()`: `cpu_model()=OtherX86`, `expects_amx=false`). On such hosts `x86-64-v4` or `native` is the correct pick; the probe was run with `x86-64-v4`, so its numbers stand. +**SPR vs EMR — resolved (2026-09-02, operator refinement + ndarray git history):** +not the enablement, the DETECTION. Pre-PR-#217 (`src/simd_caps.rs` @ `bdf243cc`, +2026-06-13) detected AMX by CPUID feature bits alone (`amx_tile`/`amx_int8`/ +`amx_bf16`/`amx_fp16`, EDX bits 24/25/22) — no XCR0 gate, no model table, and +the `arch_prctl` issued on syscall 157, so it always failed. PR #217 +(`e563fdcd`, 2026-06-14) replaced it with the four-gate detector (CPUID + +OSXSAVE + XCR0 + `arch_prctl` 158) PLUS the CPUID model table (`CpuModel`: +SPR 0x8F / EMR 0xCF / GNR 0xAD,0xAE / SRF 0xAF), added to tell "no silicon" +from "not OS-enabled". On SPR the old detector said *present* while nothing +ever executed (every tile test early-returned — Gotcha 9); on EMR the new +detector said *present AND enabled* and tiles ran. The `arch_prctl` grant is +the same on both; what differed was the detection code, and the change landed +on EMR silicon. `amx-enablement-and-kernel.md` §2 says this ("EMR was simply +the host where gate 4 got fixed first"); the operator's "detected differently" +is the same fact from the outside. Minor inconsistency noticed, not chased: +`cpu_ops.rs:186` says "Linux 5.19+", the doc says "5.16+". + Storage detail that held: with `dy` stored REVERSED, the anti-diagonal walk is forward in `i`, so k-buffers, `dx` and `dy` are all contiguous slices — **no gather**, exactly as predicted by "A1 determines A2's shape". From 83b6539a6bed3a069e8952fdf3dafa02a04efa0a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 20:25:33 +0000 Subject: [PATCH 6/8] =?UTF-8?q?jc:=20gate=20goursat=5Fsubstrate=5Fprobe=20?= =?UTF-8?q?on=20hambly-lyons=20=E2=80=94=20CI=20builds=20examples=20with?= =?UTF-8?q?=20default=20features?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI failure on #1140 (Five-Pillar Substrate Proof): error[E0432]: unresolved import `sigker` --> examples/goursat_substrate_probe.rs:21:5 sigker is an OPTIONAL dependency of jc, linked only under --features hambly-lyons. The probe imports sigker::signature_kernel_pde as its A0 reference unconditionally, and CI compiles examples with default features, so the import is a hard E0432 there — not a warning. My local runs all passed the feature explicitly, which is why it never surfaced. Fix: `required-features = ["hambly-lyons"]` on the [[example]] entry. Cargo skips the example when the feature is off instead of failing. Verified both shapes locally in release: default features, --examples ........ rc=0 (probe skipped) --features hambly-lyons, the probe .. rc=0 (builds) No code change to the probe; no change to the measurements. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp --- crates/jc/Cargo.toml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/crates/jc/Cargo.toml b/crates/jc/Cargo.toml index 2a5bab21e..8657b4d21 100644 --- a/crates/jc/Cargo.toml +++ b/crates/jc/Cargo.toml @@ -38,6 +38,14 @@ lance-graph-contract = { path = "../lance-graph-contract" } p64 = { path = "../../../ndarray/crates/p64" } neural-debug = { path = "../neural-debug" } +[[example]] +# A0/A1/A2 Goursat probe (TD-PILLAR11-SCIENTIFIC-LOOPS-BYPASS-NDARRAY-SIMD-1). +# Imports `sigker::signature_kernel_pde` as the A0 reference, so it only +# exists when Pillar 11 is on — CI builds examples with default features and +# an unconditional `use sigker` there is a hard E0432, not a warning. +name = "goursat_substrate_probe" +required-features = ["hambly-lyons"] + [[example]] name = "prove_it" From 3dcba9802d4a426cddf56d5a3e7e5963fd2a2f4c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 20:26:30 +0000 Subject: [PATCH 7/8] =?UTF-8?q?jc:=20rustfmt=20the=20Goursat=20probe=20?= =?UTF-8?q?=E2=80=94=20style.yml=20runs=20`cargo=20fmt=20--check`=20on=20j?= =?UTF-8?q?c?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I hand-formatted three one-liners rustfmt expands (a two-element vec! in path(), the two boundary `if`s, and the lo > hi early-continue). Never ran `cargo fmt` on the file; the style workflow does (.github/workflows/style.yml :226, `cargo fmt --manifest-path crates/jc/Cargo.toml -- --check`), so this would have been the NEXT red check after the sigker one. Whitespace only. Probe rebuilt under --features hambly-lyons: rc=0. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp --- crates/jc/examples/goursat_substrate_probe.rs | 45 ++++++++++++++----- 1 file changed, 35 insertions(+), 10 deletions(-) diff --git a/crates/jc/examples/goursat_substrate_probe.rs b/crates/jc/examples/goursat_substrate_probe.rs index 6d385237f..738da9474 100644 --- a/crates/jc/examples/goursat_substrate_probe.rs +++ b/crates/jc/examples/goursat_substrate_probe.rs @@ -27,7 +27,10 @@ fn path(n: usize) -> Vec> { (0..=n) .map(|i| { let t = i as f64 / n as f64; - vec![t + 0.05 * (260.0 * t).cos(), 0.5 * t + 0.05 * (260.0 * t).sin()] + vec![ + t + 0.05 * (260.0 * t).cos(), + 0.5 * t + 0.05 * (260.0 * t).sin(), + ] }) .collect() } @@ -97,12 +100,20 @@ fn goursat_wavefront(x: &[Vec], y: &[Vec]) -> f64 { for d in 2..(n + m - 1) { // Boundaries on this diagonal: k[0][d] and k[d][0] are 1. - if d < m { cur[0] = 1.0; } - if d < n { cur[d] = 1.0; } + if d < m { + cur[0] = 1.0; + } + if d < n { + cur[d] = 1.0; + } // Interior rows: i >= 1, j = d - i >= 1, i <= n-1, j <= m-1. let lo = 1usize.max(d.saturating_sub(m - 1)); let hi = (d - 1).min(n - 1); - if lo > hi { std::mem::swap(&mut prev2, &mut prev1); std::mem::swap(&mut prev1, &mut cur); continue; } + if lo > hi { + std::mem::swap(&mut prev2, &mut prev1); + std::mem::swap(&mut prev1, &mut cur); + continue; + } // dyr index for row i is (m-1-d)+i; the difference may be negative but the sum is not. let base = (m - 1).wrapping_sub(d); let mut i = lo; @@ -145,17 +156,31 @@ fn main() { ); for &n in &[256usize, 1024, 2048, 4096] { let (x, y) = (path(n), path(n)); - let t = Instant::now(); let a0 = signature_kernel_pde(&x, &y); let s0 = t.elapsed().as_secs_f64(); - let t = Instant::now(); let a1 = goursat_flat(&x, &y); let s1 = t.elapsed().as_secs_f64(); - let t = Instant::now(); let a2 = goursat_wavefront(&x, &y); let s2 = t.elapsed().as_secs_f64(); + let t = Instant::now(); + let a0 = signature_kernel_pde(&x, &y); + let s0 = t.elapsed().as_secs_f64(); + let t = Instant::now(); + let a1 = goursat_flat(&x, &y); + let s1 = t.elapsed().as_secs_f64(); + let t = Instant::now(); + let a2 = goursat_wavefront(&x, &y); + let s2 = t.elapsed().as_secs_f64(); let exact = a0.to_bits() == a1.to_bits(); let rel = ((a1 - a2) / a1).abs(); println!( "{:>6} {s0:>10.4} {s1:>10.4} {s2:>10.4} {:>8.2}x {:>8.2}x {:>12} {rel:>12.3e}", - n + 1, s0 / s1, s1 / s2, if exact { "bit-exact" } else { "DIFFERS" } + n + 1, + s0 / s1, + s1 / s2, + if exact { "bit-exact" } else { "DIFFERS" } + ); + assert!( + exact, + "A0 != A1 at n={n}: {a0:e} vs {a1:e} — storage change altered the result" ); - assert!(exact, "A0 != A1 at n={n}: {a0:e} vs {a1:e} — storage change altered the result"); } println!("\nA0 = A1 bit-exact at every size (only storage changed)."); - println!("A1 <-> A2 differ by fused vs. separate rounding of c·diag — the predeclared tolerance."); + println!( + "A1 <-> A2 differ by fused vs. separate rounding of c·diag — the predeclared tolerance." + ); } From 2becf3c3a67557045b3fb0339111ce97c45eb10c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 20:27:47 +0000 Subject: [PATCH 8/8] =?UTF-8?q?jc:=20clippy-clean=20the=20Goursat=20probe?= =?UTF-8?q?=20=E2=80=94=20needless=5Frange=5Floop=20in=20increments()?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `for a in 0..dim { out.push(w[1][a] - w[0][a]) }` -> a zip/map extend. The only clippy finding attributable to the probe; the other hits under -D warnings are pre-existing in jc's lib (unusual_byte_groupings in dueker_zoubouloglou.rs / ewa_sandwich*.rs), not this PR's, and no workflow lints jc — left alone. The probe's own A0 = A1 bit-exact assertion re-run after the change: holds at every size, so the increment arithmetic is unchanged. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp --- crates/jc/examples/goursat_substrate_probe.rs | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/crates/jc/examples/goursat_substrate_probe.rs b/crates/jc/examples/goursat_substrate_probe.rs index 738da9474..e47840f8c 100644 --- a/crates/jc/examples/goursat_substrate_probe.rs +++ b/crates/jc/examples/goursat_substrate_probe.rs @@ -40,9 +40,7 @@ fn increments(x: &[Vec]) -> (Vec, usize) { let dim = x[0].len(); let mut out = Vec::with_capacity((x.len() - 1) * dim); for w in x.windows(2) { - for a in 0..dim { - out.push(w[1][a] - w[0][a]); - } + out.extend(w[1].iter().zip(&w[0]).map(|(next, prev)| next - prev)); } (out, dim) }