From 68fcfbc798ea2758ea162ce46feadd69cc6ea020 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 02:55:42 +0000 Subject: [PATCH] docs: backfill W1.5 board entries and correct the stale W1.5 contract section MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two hygiene gaps, both found while closing out the #293/#294/#295 arc. 1. Board backfill. `.claude/blackboard.md` and `.claude/board/AGENT_LOG.md` contained ZERO mentions of the W1.5 signature primitives — all three PRs landed with no board entry, in violation of CLAUDE.md's Agent Protocol ("write decisions to the blackboard, not just to chat"). A session reading the blackboard would not have known signature_pde or randomized_signature existed. Entries added for #293, #294 and #295, carrying the decisions, the measured numbers, and the loose ends. 2. Contract-section correction. The W1.5 section of `.claude/knowledge/vertical-simd-consumer-contract.md` still described all three primitives as future work "gated on jc Pillar 11 activation". That gate opened 2026-05-07 (lance-graph jc/src/lib.rs:26), and #6 and #7 have shipped. The section is regraded in place with the workspace's existing ⊘-blockquote convention: original sketches kept verbatim, each followed by a correction block, rather than silently rewritten. The correction worth carrying forward: this doc's API sketches have now been wrong the same way TWICE. Both #6 and #7 were sketched as f32/F32x16 while the real consumer (lance-graph crates/sigker) is f64/Vec, so both shipped on F64x8. #294 additionally found the sketch wrong about buffer ownership (projections are materialized once per encoder and reused, not re-derived per step) and about shape (k is runtime, so the hot path is a k×k GEMV, not a single-register update). W1.5-#8's `I16x16` sketch is therefore marked UNVERIFIED — 2-for-2 miss rate — and must be read off crates/sigker/src/log_signature.rs before any implementation starts. Also recorded: W1.5-#8 is unblocked, not deferred; and #295 merged ~5s after leaving draft, so its 100% docstring figure is a manual /// scan rather than bot-confirmed. Markdown only — no source, tests, or build files touched. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016WkNBjHc2e3zuyz9i8qJEv --- .claude/blackboard.md | 63 ++++++++++++++++++ .claude/board/AGENT_LOG.md | 30 +++++++++ .../vertical-simd-consumer-contract.md | 66 +++++++++++++++++-- 3 files changed, 155 insertions(+), 4 deletions(-) diff --git a/.claude/blackboard.md b/.claude/blackboard.md index 6cc4e527..df066b2f 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -3,6 +3,69 @@ > **Read this first.** The "Polyglot Notebook" architecture below is a > separate/older program, not the current epoch. +## 2026-09-04 (latest) — W1.5 signature primitives: PR #293/#294/#295 landed, no board entry until now + +Three merged PRs closing W1.5 signature-kernel work items went unrecorded on +this blackboard — corrected here. + +**PR #293 — `hpc: signature_pde_sweep`** (merged), closes W1.5-#6 +`TD-NDARRAY-SIMD-SIGNATURE-PDE-SWEEP`. Signature kernel `` via the +Goursat PDE, anti-diagonal SIMD wavefront. **Correction it recorded:** the +consumer-contract doc sketched `f32`/`F32x16`; sigker is actually +`f64`/`Vec`. **Now wired:** lance-graph `crates/sigker/src/kernel.rs:35` +does `use ndarray::hpc::signature_pde::signature_pde_sweep;`. + +**PR #294 — `hpc: randomized_signature_sweep`** (merged), closes W1.5-#7 +`TD-NDARRAY-SIMD-RANDOMIZED-PROJECTION`. Cuchiero-Schmocker-Teichmann +randomized-signature recurrence on `F64x8`. New file +`src/hpc/randomized_signature.rs` + `examples/randomized_signature_bench.rs`. +Public API: `randomized_signature_sweep`, `randomized_signature_sweep_with`, +`randomized_signature_step`, `const INCREMENT_EPSILON = 1e-15`. Same +lane-type correction as #293 (doc sketched `F32x16`; real consumer is f64 → +`F64x8`); doc also wrongly sketched per-step re-derived Gaussians and a +single-register update — real shape is materialize-once buffers + runtime-k +`k×k` GEMV + axpy, `O(T·d·k²)`. Built only on already-parity-confirmed +`F64x8` methods (splat, from_slice, mul_add, reduce_sum, copy_to_slice) → zero +new arch-specific code, zero `unsafe`. [MEASURED] 2.00x @ k=32 up to 3.79x @ +k=512 vs scalar; max rel err 1e-14; cross-backend contract is 1e-9 relative +tolerance, NOT bit-equality (reduce_sum order differs per backend). Second +commit `c129662` fixed a real bug CodeRabbit caught: the ragged-path guard +used `debug_assert_eq!`, which compiles out under `--release`; a wider later +path point would silently truncate and return a signature for the WRONG +path. Switched to `assert_eq!` + a `should_panic` test proving it fires in +release. **Loose end:** sigker's `RandomizedSignatureBuilder::encode` does +NOT yet delegate — still its own scalar loop +(lance-graph `crates/sigker/src/randomized.rs:95`). Wiring is in flight this +session. + +**PR #295 — `hpc: docstring the randomized_signature tests, bench, and e2e +pipeline tests`** (merged, master `183c324`). Comments only, no behavioural +change. Motivation: CodeRabbit's docstring-coverage check on #294 reported +61.76% against an 80% threshold, scoped to functions touched by that diff (34 +functions / 3 files) — gap was entirely test + bench code; all 3 public fns +and all 4 private compute helpers were already documented. Commit 1: +docstringed SplitMix64 methods, `wiggly_path`, `assert_matches_reference`, 10 +of 11 test bodies, 8 of 9 bench fns in `src/hpc/randomized_signature.rs` + +`examples/randomized_signature_bench.rs`. Commit 2: docstringed the 7 +`e2e_tests` pipeline tests in `src/hpc/mod.rs` (outside the check's scope — +#294's diff only added a `pub mod` line to that file — but the last +undocumented fns in the hpc surface). Result: 42/42 functions documented +across the three files. CI green: 13 non-skipped jobs incl. +tests/{stable,beta,1.97.1}, clippy/1.97.1, format/stable, wasm-simd, +neon-simd, tier4-avx512, nostd/thumbv6m. **Caveat worth recording:** the PR +was merged ~5s after leaving draft, so CodeRabbit never re-ran the coverage +check — the 100% figure is verified by a strict `///` scan, not by the bot's +own (looser) heuristic. + +**Loose ends (all three PRs):** +- sigker randomized-signature wiring not yet landed (in flight, see #294). +- W1.5-#8 `TD-NDARRAY-SIMD-LYNDON-PACK` is the last unbuilt W1.5 primitive. + Its gate (jc Pillar 11, Hambly-Lyons) IS activated — `jc/src/lib.rs:26` + says "Pillar 11 activated 2026-05-07" — so #8 is unblocked, not deferred. +- The consumer-contract doc's W1.5 lane-type sketches have now been wrong + twice (both #293 and #294). #8's `I16x16` sketch should be treated as + unverified until checked against the real consumer. + ## 2026-09-01 (latest) — Pillar-11 lattice lane: BIT-EXACT i128 lattice signature + Hambly–Lyons Thm 5/6 certificate `src/hpc/pillar/lattice_signature.rs` (feature `pillar`). For unit-step diff --git a/.claude/board/AGENT_LOG.md b/.claude/board/AGENT_LOG.md index 6df44925..e3b7d48a 100644 --- a/.claude/board/AGENT_LOG.md +++ b/.claude/board/AGENT_LOG.md @@ -5,6 +5,36 @@ > **Spawn protocol:** every agent reads this file before starting, > appends one entry on completion via `tee -a`. +## 2026-09-04 — main thread — W1.5 signature primitives (#293/#294/#295) board-hygiene backfill + +**PRs:** #293 (merged), #294 (merged, `c129662` follow-up), #295 (merged, master `183c324`) +**Closes:** W1.5-#6 `TD-NDARRAY-SIMD-SIGNATURE-PDE-SWEEP`, W1.5-#7 +`TD-NDARRAY-SIMD-RANDOMIZED-PROJECTION` + +Three PRs landed with zero mentions in `.claude/blackboard.md` or this log — +backfilled per the Agent Protocol ("write decisions to the blackboard, not +just to chat"). No code changes this session; hygiene-only. Full detail in +`.claude/blackboard.md` 2026-09-04 entry — summary here: + +| PR | File(s) | Outcome | +|---|---|---| +| #293 | `src/hpc/signature_pde.rs` | Goursat-PDE signature kernel, anti-diagonal SIMD wavefront, f64/`F64x8` (doc sketch of f32/`F32x16` corrected); now consumed by lance-graph `crates/sigker/src/kernel.rs:35` | +| #294 | `src/hpc/randomized_signature.rs`, `examples/randomized_signature_bench.rs` | Cuchiero-Schmocker-Teichmann randomized-signature recurrence, `F64x8`-only (zero new arch code, zero `unsafe`); 2.00x–3.79x vs scalar; follow-up `c129662` fixed a real `debug_assert_eq!`-compiles-out-in-release bug on the ragged-path guard | +| #295 | same 3 files, docs only | Docstring coverage 61.76%→100% (42/42 fns) after CodeRabbit's scoped-diff coverage check flagged #294; caveat: merged before the bot could re-verify, so the 100% figure is a manual `///` scan, not bot-confirmed | + +**Loose ends carried forward:** sigker `RandomizedSignatureBuilder::encode` +still a standalone scalar loop, not yet delegating to +`randomized_signature_sweep` (in flight per operator note); W1.5-#8 +`TD-NDARRAY-SIMD-LYNDON-PACK` is unblocked (jc Pillar 11 activated +2026-05-07) but unbuilt — its `I16x16` lane-type sketch in the +consumer-contract doc should be treated as unverified, since both #293 and +#294 corrected an analogous sketch. + +**Verification:** none run this session (edit-only, hygiene backfill; no +`.rs` files touched). + +--- + ## 2026-07-29 — main thread — oracle repair + chacha20 reach + BLAKE3 gap **PRs:** #264 (merged), #265 (merged, `58521c3`) diff --git a/.claude/knowledge/vertical-simd-consumer-contract.md b/.claude/knowledge/vertical-simd-consumer-contract.md index 8126c974..3ab3bf1f 100644 --- a/.claude/knowledge/vertical-simd-consumer-contract.md +++ b/.claude/knowledge/vertical-simd-consumer-contract.md @@ -226,9 +226,25 @@ impl U64x4 { --- -## W1.5 — DEFERRED primitives (gated on `lance-graph:crates/sigker` certification) - -Three more primitives are queued behind a certification gate. `crates/sigker` is `lance-graph`'s path-signature codec — it's pure-scalar Rust today (zero raw intrinsics, zero ndarray dep), and is positioned as the **Index-regime third encoding lane** alongside palette-distance (bgz17) and NSM tiling (deepnsm). It explicitly bypasses the `I-NOISE-FLOOR-JIRAK` iron rule (Jirak 2016 Berry-Esseen for weak-dependence data) via Hambly-Lyons 2010 path-signature uniqueness. +## W1.5 — sigker primitives (gate now OPEN; #6 and #7 SHIPPED) + +> **⊘ CORRECTED (2026-09-04) — this section originally read as three +> future/deferred primitives, gated on a certification that had not yet +> happened. That framing is stale: the gate opened 2026-05-07 +> (`lance-graph:crates/jc/src/lib.rs:26`, "Pillar 11 activated +> 2026-05-07"; `jc/src/hambly_lyons.rs` is a live module, `pub mod +> hambly_lyons;` at `jc/lib.rs:37`), and W1.5-#6 and W1.5-#7 have since +> shipped (ndarray PR #293, PR #294). The original sketches below are +> kept verbatim, each followed by a correction block, rather than +> silently rewritten — see the standing note at the end of this section +> for why every remaining sketch (#8) must be treated as unverified.** + +Three primitives were queued behind a certification gate. `crates/sigker` is `lance-graph`'s path-signature codec — it's pure-scalar Rust today (zero raw intrinsics, zero ndarray dep), and is positioned as the **Index-regime third encoding lane** alongside palette-distance (bgz17) and NSM tiling (deepnsm). It explicitly bypasses the `I-NOISE-FLOOR-JIRAK` iron rule (Jirak 2016 Berry-Esseen for weak-dependence data) via Hambly-Lyons 2010 path-signature uniqueness. + +> **⊘ CORRECTED:** the paragraph above and the "When `jc Pillar 11`… lights +> up" sentence below described the gate as future-conditional. **The gate is +> OPEN as of 2026-05-07.** `crates/sigker` is also no longer purely scalar — +> its consumer wiring for #6 is live (see #6 below). When `jc Pillar 11` (Hambly-Lyons signature uniqueness on lance-graph paths) activates and sigker is benchmarked at production carrier widths, the W1.5 queue lights up: @@ -249,15 +265,57 @@ where 2D banded grid sweep; closure-parameterized kernel evaluator per step. +> **⊘ CORRECTED — SHIPPED (ndarray PR #293).** The sketch above is WRONG on +> lane type and is superseded, not merely deferred. Real state: +> - Shipped as `ndarray::hpc::signature_pde::signature_pde_sweep`, an +> anti-diagonal SIMD wavefront sweep. +> - **Lane type is f64-based, NOT `F32x16` as sketched.** The real +> consumer (`lance-graph crates/sigker`) works in `f64`/`Vec`, so the +> shipped primitive is built on `F64x8`. +> - **Consumer wired:** `lance-graph crates/sigker/src/kernel.rs:35` imports +> it directly; `sigker`'s `Cargo.toml` now carries ndarray as a mandatory +> path dep (no longer the "zero ndarray dep" state described above). + ### W1.5-#7 — `TD-NDARRAY-SIMD-RANDOMIZED-PROJECTION` Cuchiero-Schmocker-Teichmann (2021) randomized signatures: Gaussian random-matrix-vector update with `F32x16` state. Same closure-batch shape as W1a-#1, different lane type. +> **⊘ CORRECTED — SHIPPED (ndarray PR #294).** The sketch above is WRONG on +> lane type AND on data ownership. Real state: +> - Shipped as `ndarray::hpc::randomized_signature`, exposing +> `randomized_signature_sweep` / `_sweep_with` / `_step`, plus +> `INCREMENT_EPSILON = 1e-15`. +> - **Lane type is `F64x8`, not `F32x16`.** +> - **Ownership model was wrong too:** the sketch implied Gaussian entries +> re-derived per step from `(seed, depth)`. In reality the projections are +> materialized ONCE per encoder instance (seeded SplitMix64 + Box-Muller) +> and reused across every path and step — so the primitive must CONSUME +> caller-owned buffers, not generate them internally. +> - **`k` is a runtime value** (32…4096 in the consumer's own tests), not a +> fixed lane width — the hot path is a `k×k` GEMV plus an axpy per path +> dimension, O(T·d·k²), not a single-register lane update. +> - **Consumer NOT yet wired:** `lance-graph crates/sigker/src/randomized.rs:95` +> `RandomizedSignatureBuilder::encode` still runs its own scalar loop. +> Wiring is in flight in a parallel task as of this correction — treat as +> in-flight, not done. + ### W1.5-#8 — `TD-NDARRAY-SIMD-LYNDON-PACK` Log-signature compression in the Lyndon basis of the free Lie algebra (7-13× compression, lossless). Pack/unpack primitives on `I16x16` state with combinatorial-index awareness. -**No code needed today for W1.5.** Mentioned here so W1a additions are designed broad enough to compose with these later (in particular: the closure-batch shape introduced in W1a-#1 is the foundation for W1.5-#7). +> **⊘ CORRECTED — still unbuilt, but NO LONGER GATED** (Pillar 11 is active, +> see above). **The `I16x16` state sketch above is UNVERIFIED against the +> real consumer** (`lance-graph crates/sigker/src/log_signature.rs`) — the +> equivalent sketches for #6 and #7 were BOTH wrong on lane type (2-for-2 +> miss rate). Do not implement from this sketch. Read the actual consumer +> source first and confirm the real lane type before writing any code. + +**Standing note (2026-09-04):** the pattern across #6 and #7 is that this +doc's API sketches predate the consumer code and drift from it — both +missed the lane type, and #7 also missed the ownership model and the +runtime-`k` shape. Treat every remaining sketch in this section as a +starting hypothesis to verify against `lance-graph crates/sigker`, never as +a spec. ---