Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .claude/blackboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,69 @@
> **Read this first.** The "Polyglot Notebook" architecture below is a
> separate/older program, not the current epoch.

## 2026-09-04 (latest) — W1.5 signature primitives: PR #293/#294/#295 landed, no board entry until now

Three merged PRs closing W1.5 signature-kernel work items went unrecorded on
this blackboard — corrected here.

**PR #293 — `hpc: signature_pde_sweep`** (merged), closes W1.5-#6
`TD-NDARRAY-SIMD-SIGNATURE-PDE-SWEEP`. Signature kernel `<S(X),S(Y)>` via the
Goursat PDE, anti-diagonal SIMD wavefront. **Correction it recorded:** the
consumer-contract doc sketched `f32`/`F32x16`; sigker is actually
`f64`/`Vec<f64>`. **Now wired:** lance-graph `crates/sigker/src/kernel.rs:35`
does `use ndarray::hpc::signature_pde::signature_pde_sweep;`.

**PR #294 — `hpc: randomized_signature_sweep`** (merged), closes W1.5-#7
`TD-NDARRAY-SIMD-RANDOMIZED-PROJECTION`. Cuchiero-Schmocker-Teichmann
randomized-signature recurrence on `F64x8`. New file
`src/hpc/randomized_signature.rs` + `examples/randomized_signature_bench.rs`.
Public API: `randomized_signature_sweep`, `randomized_signature_sweep_with`,
`randomized_signature_step`, `const INCREMENT_EPSILON = 1e-15`. Same
lane-type correction as #293 (doc sketched `F32x16`; real consumer is f64 →
`F64x8`); doc also wrongly sketched per-step re-derived Gaussians and a
single-register update — real shape is materialize-once buffers + runtime-k
`k×k` GEMV + axpy, `O(T·d·k²)`. Built only on already-parity-confirmed
`F64x8` methods (splat, from_slice, mul_add, reduce_sum, copy_to_slice) → zero
new arch-specific code, zero `unsafe`. [MEASURED] 2.00x @ k=32 up to 3.79x @
k=512 vs scalar; max rel err 1e-14; cross-backend contract is 1e-9 relative
tolerance, NOT bit-equality (reduce_sum order differs per backend). Second
commit `c129662` fixed a real bug CodeRabbit caught: the ragged-path guard
used `debug_assert_eq!`, which compiles out under `--release`; a wider later
path point would silently truncate and return a signature for the WRONG
path. Switched to `assert_eq!` + a `should_panic` test proving it fires in
release. **Loose end:** sigker's `RandomizedSignatureBuilder::encode` does
NOT yet delegate — still its own scalar loop
(lance-graph `crates/sigker/src/randomized.rs:95`). Wiring is in flight this
session.

**PR #295 — `hpc: docstring the randomized_signature tests, bench, and e2e
pipeline tests`** (merged, master `183c324`). Comments only, no behavioural
change. Motivation: CodeRabbit's docstring-coverage check on #294 reported
61.76% against an 80% threshold, scoped to functions touched by that diff (34
functions / 3 files) — gap was entirely test + bench code; all 3 public fns
and all 4 private compute helpers were already documented. Commit 1:
docstringed SplitMix64 methods, `wiggly_path`, `assert_matches_reference`, 10
of 11 test bodies, 8 of 9 bench fns in `src/hpc/randomized_signature.rs` +
`examples/randomized_signature_bench.rs`. Commit 2: docstringed the 7
`e2e_tests` pipeline tests in `src/hpc/mod.rs` (outside the check's scope —
#294's diff only added a `pub mod` line to that file — but the last
undocumented fns in the hpc surface). Result: 42/42 functions documented
across the three files. CI green: 13 non-skipped jobs incl.
tests/{stable,beta,1.97.1}, clippy/1.97.1, format/stable, wasm-simd,
neon-simd, tier4-avx512, nostd/thumbv6m. **Caveat worth recording:** the PR
was merged ~5s after leaving draft, so CodeRabbit never re-ran the coverage
check — the 100% figure is verified by a strict `///` scan, not by the bot's
own (looser) heuristic.

**Loose ends (all three PRs):**
- sigker randomized-signature wiring not yet landed (in flight, see #294).
- W1.5-#8 `TD-NDARRAY-SIMD-LYNDON-PACK` is the last unbuilt W1.5 primitive.
Its gate (jc Pillar 11, Hambly-Lyons) IS activated — `jc/src/lib.rs:26`
says "Pillar 11 activated 2026-05-07" — so #8 is unblocked, not deferred.
- The consumer-contract doc's W1.5 lane-type sketches have now been wrong
twice (both #293 and #294). #8's `I16x16` sketch should be treated as
unverified until checked against the real consumer.

## 2026-09-01 (latest) — Pillar-11 lattice lane: BIT-EXACT i128 lattice signature + Hambly–Lyons Thm 5/6 certificate

`src/hpc/pillar/lattice_signature.rs` (feature `pillar`). For unit-step
Expand Down
30 changes: 30 additions & 0 deletions .claude/board/AGENT_LOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,36 @@
> **Spawn protocol:** every agent reads this file before starting,
> appends one entry on completion via `tee -a`.

## 2026-09-04 — main thread — W1.5 signature primitives (#293/#294/#295) board-hygiene backfill

**PRs:** #293 (merged), #294 (merged, `c129662` follow-up), #295 (merged, master `183c324`)
**Closes:** W1.5-#6 `TD-NDARRAY-SIMD-SIGNATURE-PDE-SWEEP`, W1.5-#7
`TD-NDARRAY-SIMD-RANDOMIZED-PROJECTION`

Three PRs landed with zero mentions in `.claude/blackboard.md` or this log —
backfilled per the Agent Protocol ("write decisions to the blackboard, not
just to chat"). No code changes this session; hygiene-only. Full detail in
`.claude/blackboard.md` 2026-09-04 entry — summary here:

| PR | File(s) | Outcome |
|---|---|---|
| #293 | `src/hpc/signature_pde.rs` | Goursat-PDE signature kernel, anti-diagonal SIMD wavefront, f64/`F64x8` (doc sketch of f32/`F32x16` corrected); now consumed by lance-graph `crates/sigker/src/kernel.rs:35` |
| #294 | `src/hpc/randomized_signature.rs`, `examples/randomized_signature_bench.rs` | Cuchiero-Schmocker-Teichmann randomized-signature recurrence, `F64x8`-only (zero new arch code, zero `unsafe`); 2.00x–3.79x vs scalar; follow-up `c129662` fixed a real `debug_assert_eq!`-compiles-out-in-release bug on the ragged-path guard |
| #295 | same 3 files, docs only | Docstring coverage 61.76%→100% (42/42 fns) after CodeRabbit's scoped-diff coverage check flagged #294; caveat: merged before the bot could re-verify, so the 100% figure is a manual `///` scan, not bot-confirmed |

**Loose ends carried forward:** sigker `RandomizedSignatureBuilder::encode`
still a standalone scalar loop, not yet delegating to
`randomized_signature_sweep` (in flight per operator note); W1.5-#8
`TD-NDARRAY-SIMD-LYNDON-PACK` is unblocked (jc Pillar 11 activated
2026-05-07) but unbuilt — its `I16x16` lane-type sketch in the
consumer-contract doc should be treated as unverified, since both #293 and
#294 corrected an analogous sketch.

**Verification:** none run this session (edit-only, hygiene backfill; no
`.rs` files touched).

---

## 2026-07-29 — main thread — oracle repair + chacha20 reach + BLAKE3 gap

**PRs:** #264 (merged), #265 (merged, `58521c3`)
Expand Down
66 changes: 62 additions & 4 deletions .claude/knowledge/vertical-simd-consumer-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,9 +226,25 @@ impl U64x4 {

---

## W1.5 — DEFERRED primitives (gated on `lance-graph:crates/sigker` certification)

Three more primitives are queued behind a certification gate. `crates/sigker` is `lance-graph`'s path-signature codec — it's pure-scalar Rust today (zero raw intrinsics, zero ndarray dep), and is positioned as the **Index-regime third encoding lane** alongside palette-distance (bgz17) and NSM tiling (deepnsm). It explicitly bypasses the `I-NOISE-FLOOR-JIRAK` iron rule (Jirak 2016 Berry-Esseen for weak-dependence data) via Hambly-Lyons 2010 path-signature uniqueness.
## W1.5 — sigker primitives (gate now OPEN; #6 and #7 SHIPPED)

> **⊘ CORRECTED (2026-09-04) — this section originally read as three
> future/deferred primitives, gated on a certification that had not yet
> happened. That framing is stale: the gate opened 2026-05-07
> (`lance-graph:crates/jc/src/lib.rs:26`, "Pillar 11 activated
> 2026-05-07"; `jc/src/hambly_lyons.rs` is a live module, `pub mod
> hambly_lyons;` at `jc/lib.rs:37`), and W1.5-#6 and W1.5-#7 have since
> shipped (ndarray PR #293, PR #294). The original sketches below are
> kept verbatim, each followed by a correction block, rather than
> silently rewritten — see the standing note at the end of this section
> for why every remaining sketch (#8) must be treated as unverified.**

Three primitives were queued behind a certification gate. `crates/sigker` is `lance-graph`'s path-signature codec — it's pure-scalar Rust today (zero raw intrinsics, zero ndarray dep), and is positioned as the **Index-regime third encoding lane** alongside palette-distance (bgz17) and NSM tiling (deepnsm). It explicitly bypasses the `I-NOISE-FLOOR-JIRAK` iron rule (Jirak 2016 Berry-Esseen for weak-dependence data) via Hambly-Lyons 2010 path-signature uniqueness.

> **⊘ CORRECTED:** the paragraph above and the "When `jc Pillar 11`… lights
> up" sentence below described the gate as future-conditional. **The gate is
> OPEN as of 2026-05-07.** `crates/sigker` is also no longer purely scalar —
> its consumer wiring for #6 is live (see #6 below).

When `jc Pillar 11` (Hambly-Lyons signature uniqueness on lance-graph paths) activates and sigker is benchmarked at production carrier widths, the W1.5 queue lights up:

Expand All @@ -249,15 +265,57 @@ where

2D banded grid sweep; closure-parameterized kernel evaluator per step.

> **⊘ CORRECTED — SHIPPED (ndarray PR #293).** The sketch above is WRONG on
> lane type and is superseded, not merely deferred. Real state:
> - Shipped as `ndarray::hpc::signature_pde::signature_pde_sweep`, an
> anti-diagonal SIMD wavefront sweep.
> - **Lane type is f64-based, NOT `F32x16` as sketched.** The real
> consumer (`lance-graph crates/sigker`) works in `f64`/`Vec<f64>`, so the
> shipped primitive is built on `F64x8`.
> - **Consumer wired:** `lance-graph crates/sigker/src/kernel.rs:35` imports
> it directly; `sigker`'s `Cargo.toml` now carries ndarray as a mandatory
> path dep (no longer the "zero ndarray dep" state described above).

### W1.5-#7 — `TD-NDARRAY-SIMD-RANDOMIZED-PROJECTION`

Cuchiero-Schmocker-Teichmann (2021) randomized signatures: Gaussian random-matrix-vector update with `F32x16` state. Same closure-batch shape as W1a-#1, different lane type.

> **⊘ CORRECTED — SHIPPED (ndarray PR #294).** The sketch above is WRONG on
> lane type AND on data ownership. Real state:
> - Shipped as `ndarray::hpc::randomized_signature`, exposing
> `randomized_signature_sweep` / `_sweep_with` / `_step`, plus
> `INCREMENT_EPSILON = 1e-15`.
> - **Lane type is `F64x8`, not `F32x16`.**
> - **Ownership model was wrong too:** the sketch implied Gaussian entries
> re-derived per step from `(seed, depth)`. In reality the projections are
> materialized ONCE per encoder instance (seeded SplitMix64 + Box-Muller)
> and reused across every path and step — so the primitive must CONSUME
> caller-owned buffers, not generate them internally.
> - **`k` is a runtime value** (32…4096 in the consumer's own tests), not a
> fixed lane width — the hot path is a `k×k` GEMV plus an axpy per path
> dimension, O(T·d·k²), not a single-register lane update.
> - **Consumer NOT yet wired:** `lance-graph crates/sigker/src/randomized.rs:95`
> `RandomizedSignatureBuilder::encode` still runs its own scalar loop.
> Wiring is in flight in a parallel task as of this correction — treat as
> in-flight, not done.

### W1.5-#8 — `TD-NDARRAY-SIMD-LYNDON-PACK`

Log-signature compression in the Lyndon basis of the free Lie algebra (7-13× compression, lossless). Pack/unpack primitives on `I16x16` state with combinatorial-index awareness.

**No code needed today for W1.5.** Mentioned here so W1a additions are designed broad enough to compose with these later (in particular: the closure-batch shape introduced in W1a-#1 is the foundation for W1.5-#7).
> **⊘ CORRECTED — still unbuilt, but NO LONGER GATED** (Pillar 11 is active,
> see above). **The `I16x16` state sketch above is UNVERIFIED against the
> real consumer** (`lance-graph crates/sigker/src/log_signature.rs`) — the
> equivalent sketches for #6 and #7 were BOTH wrong on lane type (2-for-2
> miss rate). Do not implement from this sketch. Read the actual consumer
> source first and confirm the real lane type before writing any code.

**Standing note (2026-09-04):** the pattern across #6 and #7 is that this
doc's API sketches predate the consumer code and drift from it — both
missed the lane type, and #7 also missed the ownership model and the
runtime-`k` shape. Treat every remaining sketch in this section as a
starting hypothesis to verify against `lance-graph crates/sigker`, never as
a spec.

---

Expand Down
Loading