diff --git a/.claude/board/AGENT_LOG.md b/.claude/board/AGENT_LOG.md index 791d1f867..b8ec2e373 100644 --- a/.claude/board/AGENT_LOG.md +++ b/.claude/board/AGENT_LOG.md @@ -1,3 +1,12 @@ +## 2026-09-05 — Sonnet harvest fleet (4 readers + 7 PR sweepers), orchestrator-consolidated → `.claude/nexgen/` + +- **Why:** operator asked how HDR popcount-stacking, rolling floor, Prozentrang buckets, Mexican hat, Shannon, proprioception, EWA sandwich and sigker could combine with ternlog mask algebra for better thresholds; then asked for the last 50 lance-graph PRs + 25 OGAR + 25 ndarray swept for synergies, epiphanies written to the board, the raw agent output kept, and an expansion plan in the T0..T3 shape "27 rooms ahead". +- **Fleet (all Sonnet, all read-only, no cargo, no GitHub writes):** 4 `Explore` readers (cascade/rolling-floor/hdr; Prozentrang+Shannon doctrine; proprioception/EWA/Mexican/16k-search/mask surface; sigker+"power kernels"), 7 `general-purpose` sweepers over fixed PR partitions (lance-graph #1126–#1175 in five tens, OGAR #274–#298, ndarray #277–#301) using `pull_request_read` get + get_files. Each agent wrote only its own return; the orchestrator is the sole writer of every board file here. +- **Outputs:** `.claude/nexgen/harvest/00-INDEX.md` + 11 verbatim reports; `.claude/nexgen/plans/nexgen-mask-histogram-thresholds-v1.md` (D-NXG-1..12, 27-room expansion table with a falsifier per row); EPIPHANIES `E-NXG-1..16`; INTEGRATION_PLANS + STATUS_BOARD rows. +- **Headline:** the exposure meter is a nested mask set (E-NXG-1); Prozentrang and bucket rollover are unbuilt (E-NXG-2/3); only `TERNLOG 0x86` survives the OGAR #298 retraction (E-NXG-7); "power kernel" does not exist (E-NXG-15). +- **Not done, on purpose:** no code, no bench, no removal or demotion of any public surface (operator ruling 2026-09-04 stands). The plan sits under `.claude/nexgen/plans/`, outside `supersession_index.py`'s scan path — recorded in the plan header. +- **Gates:** board append-only check (line counts grew), supersession index regenerated last. + ## 2026-09-03 — 5+3 COUNCIL on the canon entries that shipped unreviewed (#1154) - **Why convened:** three EPIPHANIES entries + two source-doc blocks sat on an OPEN PR that neither review bot had seen (Codex reviewed only `a1c9488e`; CodeRabbit at its org spend cap). EPIPHANIES is append-only, so this was the last moment the text was amendable rather than permanent-with-regrade — the card's "a spec whose wrong resolution silently corrupts downstream sessions (canon entries)". CI status, merge mechanics and the spend decision were explicitly EXCLUDED from the object. diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 165578e27..997527c16 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,179 @@ +## 2026-09-05 — E-NXG-1 — the exposure meter is a nested mask set + +**Status:** PROPOSAL (design reading of measured code; no code changed). +**Confidence:** High on the identity, unmeasured on the cost. + +Foveal ⊆ Near ⊆ Good ⊆ Weak (`ndarray/src/hpc/cascade.rs:162-176`) are four +row masks of 8 KB each at 65 536 rows; the histogram is four popcounts; bucket +i is `M_i ∧ ¬M_{i−1}` = `vpternlogq` immediate `AND_ANDNOT2 = 0x10` +(`ndarray/src/simd.rs`), one instruction per 512 rows. Belichtungsmesser, +Prozentrang, Mexican hat, basin and the mask slab cache are one sealed object. +Plan: `.claude/nexgen/plans/nexgen-mask-histogram-thresholds-v1.md` (D-NXG-1). +Harvest: `.claude/nexgen/harvest/01-*.md`. + +## 2026-09-05 — E-NXG-2 — Prozentrang exists only in doctrine; in code nothing ranks against a live distribution + +**Status:** FINDING (grep + read, `.claude/nexgen/harvest/02-*.md`). +**Confidence:** High. + +`observer-effect-tfpn-doctrine.md` §2 mandates `shape × rank`; D-BLW-5 designs a +16-bucket Fisher-2z histogram; `ndarray::hpc::statistics::percentile` is a batch +sort. No live rank mechanism exists. Under E-NXG-1 rank = index of the innermost +band mask containing the row — a partition point over nested masks, no sort +(D-NXG-4). + +## 2026-09-05 — E-NXG-3 — bucket rollover exists nowhere; it is a popcount test plus a mask split + +**Status:** FINDING on the absence; PROPOSAL on the mechanism. +**Confidence:** High / unmeasured. + +The only mentions are the two "lacking proper bucket rollover" module-doc +bullets in `crates/lance-graph-contract/src/legacy_outliers.rs` ("it saturates +silently. Give it rollover, or narrow it") and +`crates/lance-graph-contract/src/identity_quad.rs` (the codebook refuses to +exceed capacity "rather than saturating silently"). +Mechanism: when `popcount(M_i ∧ ¬M_{i−1})` exceeds budget, bisect that bucket on +the distance column (the one non-mask read) and mint the new boundary as a new +version-keyed mask; old masks are never rewritten (D-NXG-5). Merge on collapse is +the dual: drop a boundary, the merged mask is already `M_{i+1}`. + +## 2026-09-05 — E-NXG-4 — `mu + kσ` is the wrong operating threshold and the code already says so + +**Status:** FINDING (code cites the objection to itself). +**Confidence:** High. + +`cascade.rs:137` fixes `k=3`; `perturbation-sim/src/rolling_floor.rs:25-27` +states the σ is from a weakly-dependent sample and "significance is the Jirak +`n^(p/2−1)` rate, not a clean Gaussian tail". Under E-NXG-1 the reject floor is +the boundary at a chosen empirical rank read off cumulative popcounts; σ becomes +`RollingFloor::z()` diagnostic output (D-NXG-6). Cross-ref I-NOISE-FLOOR-JIRAK. + +## 2026-09-05 — E-NXG-5 — preheat by mask inheritance beats copying mu/σ + +**Status:** PROPOSAL. +**Confidence:** Medium (strictly more information for one AND; cost unmeasured). + +`stack_early_exit()` (`rolling_floor.rs:230-235`) preheats a cold fine tier by +copying two scalars from the coarse tier. `M_fine_domain = M_coarse[weak]` +carries the whole survivor set and the coarse histogram restricted to it, for one +`AND3` (D-NXG-7). + +## 2026-09-05 — E-NXG-6 — early exit is a popcount budget; search and alarm are one rule + +**Status:** PROPOSAL. +**Confidence:** Medium. + +First-Alarm exit (`rolling_floor.rs:239-247`) and kth-best tightening +(`holograph/src/width_16k/search.rs:483`) are one rule: +`popcount(survivors) ≤ k`. Top-k with k = floor is the rolling floor (D-NXG-8). + +## 2026-09-05 — E-NXG-7 — only `TERNLOG = FnIndex(0x86)` survives, and only because its value byte is the truth table + +**Status:** FINDING (OGAR #296/#298, lance-graph #1134/#1159). +**Confidence:** High. + +OGAR #298 retracted 0x87–0x8B because one band aliased three semantic families +(`E-SIX-SEMANTIC-FAMILIES-MUST-NOT-IMPERSONATE-EACH-OTHER-1`). INFO_GAIN / +SIGMA_TENSION / STANCE_ENTROPY return only as DERIVED reads over single-family +masks, never as minted calls. Any derived read that must AND masks from two +families is a violation at the seal. + +## 2026-09-05 — E-NXG-8 — the eight named immediates already have cognitive homes + +**Status:** FINDING (mapping of shipped code). +**Confidence:** High on the mapping. + +`AND3` = conjunctive narrowing (`lgj_hop`, shipped); `AND_ANDNOT2` = bucket / +annulus / known-false (`domain ∧ ¬result`); `MAJ3` = quorum +(`jc::quorum::pairwise_agreement_u8`, `superposition_clean` n/2 vote at +`hdr_cascade.rs:564`); `XOR3` = disagreement (`EpistemicBassin24` agree/disagree +pair, #1129), sign side only per the TWO-ALGEBRA rule; `OR2_AND` = gated +hypothesis union for the prefetch frontier. + +## 2026-09-05 — E-NXG-9 — histogram entropy is a free threshold-health signal + +**Status:** PROPOSAL over ruled primitives. +**Confidence:** Medium. + +`H(popcounts)` via `thought_atoms::normalized_entropy` (ruled home, #1153/#1154; +popcount-histogram form already at `spectroscopy/features.rs:89`). Near zero +means one band holds everything: the threshold is mis-set or rollover is +overdue. Rollover becomes entropy-timed, not budget-timed (D-NXG-9). + +## 2026-09-05 — E-NXG-10 — the Mexican hat has no certified shape + +**Status:** FINDING. +**Confidence:** High. + +`hdr_cascade.rs:128-141` is a piecewise-linear ramp; Pillar-15 +(`ndarray/src/hpc/pillar/mexican_hat.rs:111`) is DEFERRED and returns a +placeholder `passed=true`. Under E-NXG-1 excite/inhibit are two band boundaries, +κ ∈ [1.5, 3.0] is a ratio of ranks that rollover keeps in band, and unimodality +is a finite check on the discrete histogram at seal time (D-NXG-10). + +## 2026-09-05 — E-NXG-11 — the EWA sandwich is the floor's covariance, but uncalibrated + +**Status:** FINDING on the blocker; PROPOSAL on the use. +**Confidence:** High / Medium. + +Adjacent tiers are correlated (Jirak note). A 2×2 `Σ' = MΣM` per tier pair +(`contract::sigma_propagation::ewa_sandwich` ≡ `jc::ewa_sandwich`, byte-identical +per #1160) replaces "copy σ down" with PSD-safe propagation. Blocker: +`PILLAR_6_PSD_THRESHOLD` was lowered to 0.10 because `SIGMA_STEP = 0.2` reaches +denormal in <30 hops (`ewa_sandwich_2d.rs:53-58`). D-NXG-11 is Blocked on that +calibration. + +## 2026-09-05 — E-NXG-12 — proprioception violates the payload law as written + +**Status:** FINDING. +**Confidence:** High. + +`contract/src/proprioception.rs` classifies by nearest-anchor distance over an +11-dim vector — a raw scalar, the shape the payload law forbids. Under +E-NXG-1 each anchor is a band over the window; state = popcounts × rank; +`drive_ratio` cutoffs 1.0 / 1.8 become boundaries with rollover. + +## 2026-09-05 — E-NXG-13 — the ownership and iteration shapes already exist in OGAR + +**Status:** FINDING. +**Confidence:** High. + +`BasinCodebookBuilder::seal()` (OGAR #295: build, freeze, no `&mut` after) is the +slab's ownership shape; inline `[u64;3]` `CallMask` + `set_indices()` (OGAR #288) +is its iteration shape; `resolve(0) → None` (`ogar-loco/src/basin.rs`) becomes +"an absent band mask misses". A basin is a band. + +## 2026-09-05 — E-NXG-14 — sigker has no row-level mask link and should not; the link is one level up + +**Status:** FINDING on the absence; CONJECTURE on the use. +**Confidence:** High / Low. + +No mask/popcount/HDR reference exists in `crates/sigker` or the W1.5 plan +(`.claude/nexgen/harvest/04-*.md`). A signature is over a path, not a row set. +The per-tier survivor-popcount sequence is a length-4 path in ℝ⁴; +`signature_kernel_pde` on two such paths is an order-sensitive query-narrowing +similarity for the prefetch planner. Untested. + +## 2026-09-05 — E-NXG-15 — "power kernel" does not exist in either workspace + +**Status:** FINDING (grep). +**Confidence:** High. + +Only unrelated power iteration (PCA in `ndarray/bf16_test_src/main.rs:1029`, +PageRank in `arigraph/ppr.rs:30`, SVD in `bgz-tensor/src/matryoshka.rs:304`). +Recorded so the term is not re-searched. + +## 2026-09-05 — E-NXG-16 — what is not free + +**Status:** FINDING (cost ledger for E-NXG-1..15). +**Confidence:** High. + +Popcount-derived selectivity assumes operand independence (the overlap matrix +`popcount(M_a ∧ M_b)` is the fix, D-NXG-12). 16 bands × 8 KB per histogram per +version hot; a u8 rank column is the cold form. Rollover bisection reads the +distance column. Nothing is measured; `columnar_hop_bench` is the only bench in +scope. Evidence trail for all sixteen entries: `.claude/nexgen/harvest/`. + ## 2026-09-04 — E-A-SKETCH-THAT-MISSED-TWICE-WILL-MISS-A-THIRD-TIME-1 — the W1.5 lane-type sketches are 3-for-3 wrong **Status:** FINDING (read directly off `crates/sigker` source this session). diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index 35d5fd21d..88315147e 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -1,3 +1,18 @@ +## 2026-09-05 — `nexgen-mask-histogram-thresholds-v1` (PROPOSAL) — lives under `.claude/nexgen/plans/` + +`.claude/nexgen/plans/nexgen-mask-histogram-thresholds-v1.md`. D-NXG-1..12. The +Belichtungsmesser reading as a nested mask set: bands, buckets, Prozentrang, +rollover, reject floor, preheat, early exit, Mexican hat, entropy timer, EWA +covariance and the prefetch overlap matrix as reads of one sealed, +version-keyed `NestedBands` per (classid, version). §2 gives the T0..T3 +entropy shape (what each membrane stops having to name); §3 is a 27-row +"rooms ahead" table, each row a folded consequence with its own falsifier; §5 +sequences three probes before any of rooms 4–27 may leave PROPOSAL. Evidence: +`.claude/nexgen/harvest/` (11 verbatim Sonnet reports, 2026-09-05). Board: +`E-NXG-1..16`. Not in `.claude/plans/`, so invisible to the supersession +index's coverage column — stated in the plan header, not a gap to "fix" by +moving it until the folder earns a second plan. + ## 2026-09-03 — `probe-r2il-live-regfile-v1` (PROBE, GREEN 2026-08-26) — BACKFILLED INDEX ENTRY `.claude/plans/probe-r2il-live-regfile-v1.md`. The executable falsifier for diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 90680145e..8f894fc28 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -1,3 +1,23 @@ +## nexgen-mask-histogram-thresholds-v1 (PROPOSAL, 2026-09-05) + +`.claude/nexgen/plans/nexgen-mask-histogram-thresholds-v1.md`. Nothing built; +every row is Queued until PROBE-NXG-HIST-1 / ROLL-1 / FLOOR-1 (plan §5) run. + +| D-id | deliverable | status | +|---|---|---| +| D-NXG-1 | `NestedBands` sealed shape (T2), version-keyed, one owner | Queued | +| D-NXG-2 | T1 name audit: `popcount_words` present or minted; bucket = `mask_ternlog::<0x10>` by name | Queued | +| D-NXG-3 | T1 `bisect_column_by_mask` partial-popcount bisection | Queued | +| D-NXG-4 | Prozentrang = bucket index; `shape × rank` computed from the slab | Queued | +| D-NXG-5 | rollover: split on budget or entropy, merge on collapse, never rewrite | Queued | +| D-NXG-6 | rank-derived reject floor; σ demoted to diagnostic | Queued | +| D-NXG-7 | preheat by mask inheritance | Queued | +| D-NXG-8 | one early-exit rule for search and alarm | Queued | +| D-NXG-9 | histogram entropy via `thought_atoms::normalized_entropy` as rollover timer | Queued | +| D-NXG-10 | Mexican hat as two boundaries; Pillar-15 on the histogram | Queued | +| D-NXG-11 | EWA sandwich on histogram-recovered Σ | **Blocked** on Pillar-6 σ_step calibration (`ewa_sandwich_2d.rs:53-58`) | +| D-NXG-12 | overlap matrix + corrected prefetch order + mask-native quorum input | Queued | + ## probe-r2il-live-regfile-v1 (D-ids minted 2026-09-03; the plan shipped 2026-08-26) `.claude/plans/probe-r2il-live-regfile-v1.md`. The falsifier for the EXECUTABLE diff --git a/.claude/nexgen/README.md b/.claude/nexgen/README.md new file mode 100644 index 000000000..35be8fb68 --- /dev/null +++ b/.claude/nexgen/README.md @@ -0,0 +1,16 @@ +# .claude/nexgen — next-generation architecture harvest + expansion plans + +Created 2026-09-05. Two folders: + +- `harvest/` — verbatim reports from the read-only Sonnet reader/sweeper agents + spawned on 2026-09-05 (4 code/doctrine readers, 7 PR sweepers: lance-graph + #1126–#1175, OGAR #274–#298, ndarray #277–#301). These are RAW agent output, + kept as evidence; nothing in here is ruled. Each file names its scope. +- `plans/` — expansion plans derived from the harvest. Each plan carries D-ids + and is indexed in `.claude/board/INTEGRATION_PLANS.md` like any other plan. + +The consolidated epiphanies live on the board +(`.claude/board/EPIPHANIES.md`, 2026-09-05 block, `E-NXG-*`), not here. This +folder is the working set behind them. + +Reading order: `harvest/00-INDEX.md` → the plan → the board block. diff --git a/.claude/nexgen/harvest/00-INDEX.md b/.claude/nexgen/harvest/00-INDEX.md new file mode 100644 index 000000000..3513a0627 --- /dev/null +++ b/.claude/nexgen/harvest/00-INDEX.md @@ -0,0 +1,25 @@ +# Harvest index — 2026-09-05 + +Topic hunted: a per-mailbox, Lance-version-keyed slab cache of row masks + an +index mask over predicate space; ternlog mask algebra; HDR popcount-stacking +early-exit cascade (Belichtungsmesser, rolling floor, preheat, bands); +Prozentrang / bucket-rank payload law; Mexican-hat threshold curve; Shannon +entropy primitives; proprioception anchors; EWA/temporal sandwich SPD +certification; sigker signature kernels; basins (zero ladder). + +| file | agent | scope | +|---|---|---| +| `01-cascade-rolling-floor-hdr.md` | Sonnet reader | `ndarray/src/hpc/cascade.rs`, `perturbation-sim/src/rolling_floor.rs`, `holograph/src/hdr_cascade.rs` | +| `02-prozentrang-shannon-doctrine.md` | Sonnet reader | payload law, Shannon primitives census, bucket-rollover absence | +| `03-proprioception-ewa-mexican-mask-surface.md` | Sonnet reader | `proprioception.rs`, pillars 6/8/15, `width_16k/search.rs`, `simd_int_ops.rs` mask surface | +| `04-sigker-power-kernels.md` | Sonnet reader | sigker crate, W1.5 primitives, Pillar 11 gate, "power kernel" absence | +| `10-pr-sweep-lance-graph-1175-1166.md` | Sonnet sweeper | 10 PRs | +| `11-pr-sweep-lance-graph-1165-1156.md` | Sonnet sweeper | 10 PRs | +| `12-pr-sweep-lance-graph-1155-1146.md` | Sonnet sweeper | 10 PRs | +| `13-pr-sweep-lance-graph-1145-1136.md` | Sonnet sweeper | 10 PRs | +| `14-pr-sweep-lance-graph-1135-1126.md` | Sonnet sweeper | 10 PRs | +| `15-pr-sweep-ogar-298-274.md` | Sonnet sweeper | 25 PRs | +| `16-pr-sweep-ndarray-301-277.md` | Sonnet sweeper | 25 PRs | + +All agents were read-only (no edits, no cargo, no GitHub writes). Line numbers +are as of the branch heads on 2026-09-05 and decay under every prepend. diff --git a/.claude/nexgen/harvest/01-cascade-rolling-floor-hdr.md b/.claude/nexgen/harvest/01-cascade-rolling-floor-hdr.md new file mode 100644 index 000000000..50b96e10b --- /dev/null +++ b/.claude/nexgen/harvest/01-cascade-rolling-floor-hdr.md @@ -0,0 +1,47 @@ +# Reader 1 — cascade.rs, rolling_floor.rs, hdr_cascade.rs (verbatim, 2026-09-05) + +## `/home/user/ndarray/src/hpc/cascade.rs` +- `Band` enum (line 22-28): `Foveal, Near, Good, Weak, Reject` +- `expose()` (line 162-176): quarters of `threshold` t: `d<=t/4`→Foveal, `<=t/2`→Near, `<=t*3/4`→Good, `<=t`→Weak, else Reject +- `ShiftAlert` struct (line 32-38): `old_mu, new_mu, old_sigma, new_sigma, observations` +- `Cascade` state fields (line 104-109): `threshold: u64, vec_bytes, mu: f64, sigma: f64, observations: usize` +- `calibrate()` (line 137-160): threshold = `mu + 3.0*sigma` (k=3 fixed), from batch mean/var +- `observe()` (line 182-209): Welford online mu/sigma update; drift check line 198: `observations>10 && old_sigma>0.0 && |mu-old_mu| > 2.0*old_sigma` → emits `ShiftAlert` +- `recalibrate()` (line 211-215): sets mu/sigma from alert, `threshold = new_mu + 3.0*new_sigma` +- Stroke/stacking early exit: `query()` (line 218-309) — Stroke 1 uses `s1_bytes = (((vec_bytes/16).max(64)+63)&!63).min(vec_bytes)` prefix, computes `sigma_est` (binomial approx) and `sigma_pop` (empirical from `warmup_n=128.min(num_vectors)` warmup samples), `sigma = sigma_est.max(sigma_pop).max(1.0)`, reject bound `s1_reject = threshold + 3.0*sigma`; survivors carried to Stroke 2 full Hamming (line 291-306); small vectors (`vec_bytes<128`) skip cascade entirely (line 229-244) +- `PackedDatabase::cascade_query()` (line 583-647) is a 3-stroke variant with different scale factors per stroke (`estimate <= threshold + threshold/4` for stroke1, exact `threshold` for stroke2/3) +- No reservoir sampling present — warmup uses first `warmup_n=128` items directly +- Bucket/percentile mechanism: none — threshold+band based. `adaptive_resolution()` (line 527-534) is a match-based band selector by `(query_entropy, corpus_cv)`, not a bucket/percentile mechanism + +## `/home/user/lance-graph/crates/perturbation-sim/src/rolling_floor.rs` +- `FloorBand` enum (line 33-44): `Stable, Watch, Concern, Warning, Alarm` — same quarter-of-threshold scheme as `Cascade::Band` +- `weyl_over_fiedler()` (line 52-65): `Δλ.max(0.0) / λ₂.abs()`, guarded by `SPECTRAL_GAP_FLOOR` → returns `FRAGMENTATION_SENTINEL` if `|λ₂| < floor` (never NaN) +- `RollingFloor` struct (line 72-78): fields `mu: f64, sigma: f64, pub k: f64, n: usize` +- `k` default: none hardcoded — caller supplies via `RollingFloor::new(k)` / `TierFloors::new(k)`; doc (line 75) suggests 2.0 ≈ 97.7% one-sided Gaussian; tests use k=2.0 and k=3.0 +- `preheat()` (line 93-103): batch-calibrates mu/sigma from `samples: &[f64]`, sets `n = samples.len()` — how a coarser tier's floor warm-starts a colder finer-tier floor, invoked at `stack_early_exit()` line 230-235: `if self.floors[t+1].threshold() <= 0.0 { copy mu/sigma/n from floors[t] into floors[t+1] (n.max(1)) }` +- `observe()` (line 108-122): tests `band(x)==Alarm` BEFORE updating (line 109), then Welford update +- `threshold()` (line 125-127): `mu + k*sigma` +- `z()` (line 131-137): standardized exceedance `(x-mu)/sigma`, 0 if sigma<1e-12 +- `band()` (line 140-161): quarters of threshold; uncalibrated (`t<=0.0`): `x>0.0`→Alarm else Stable +- `TierFloors` (line 192-256): `floors: [RollingFloor;4]` for L1..L4 (HEEL/HIP/TWIG/LEAF) +- `stack_early_exit()` (line 224-256): accumulates `stacked += inc` per tier, preheats next-cold floor, computes band+z, calls `observe()`, early-exits at first tier whose band==Alarm (line 239-247), returning `StackResult{exit_tier, stacked, band, early: t<3, z}` +- Jirak note (line 25-27): "the σ here is from a small, weakly-dependent tier sample, so the nominal CI is approximate — significance is the Jirak `n^(p/2−1)` rate, not a clean Gaussian tail. The floor is an operating threshold, not a proof." +- No bucket/percentile (Prozentrang) mechanism present + +## `/home/user/lance-graph/crates/holograph/src/hdr_cascade.rs` +- Level architecture (line 8-42): L0 Belichtungsmesser (7-point sample, ~90% filtered) → L1 1-bit scan (~80% filtered) → L2 Stacked Popcount (early exit if impossible) → L3 Mexican Hat discrimination → optional Voyager Deep Field stacking fallback +- Constants: `DEFAULT_EXCITE=2000` (line 52), `DEFAULT_INHIBIT=5000` (line 55), `METER_POINTS: [usize;7] = [0,23,47,78,101,131,155]` (line 59, prime-spaced, `#[allow(dead_code)]`/unwired) +- `MexicanHat` struct (line 83-90): `excite: u32, inhibit: u32, inhibit_strength: f32` (default 0.5) +- `response()` (line 128-141): if `distance=4`; `slope = (h[3]-h[0])/3` over 4-slot ring (`sd_history[self.sd_idx % 4]`); `slope>10`: `threshold = (threshold + slope*20).min(5000)`; `slope<-10`: `(threshold + slope*15).max(500)` +- `should_retreat()` (line 237-239): `current_quality < ema * 0.6` +- `update_quality()` (line 232-234): `ema = 0.85*ema + 0.15*batch_quality` +- `HdrCascade` state (line 286-299): `threshold_l0: f32 (=0.8), threshold_l1: u32 (=130), threshold_l2: u32 (=3000), batch_size (=64)` +- Per-level early exit in `search()` (line 371-399): L0 `meter.definitely_far(threshold_l0)`; L1 `count_differing_words > threshold_l1`; L2 `StackedPopcount::compute_with_threshold(...)` `None` = rejected +- `search_adaptive()` (line 402-452): per batch of 64, meters first item, `dynamic_threshold` via `calculate_sweet_spot`; quality early break (line 428-434) once `batch_count>=8 && i>batch_start+16` if `should_retreat` +- `classify_signal()` (line 625-632): `(mean, sd, distance)` → `Strong/Moderate/WeakButStackable/Noise`; `_ => Noise` +- `voyager_deep_field()` (line 475-525): capped collection up to `stack_size` (not reservoir); `superposition_clean()` (line 564-605) majority vote `n/2` +- `RollingWindow` (line 764-855): ring buffer with running `sum`/`sum_sq`; `is_coherent()` (line 843-845) `cv() < threshold`; eviction subtracts oldest +- No true reservoir sampling (Algorithm R) in any of the three files diff --git a/.claude/nexgen/harvest/02-prozentrang-shannon-doctrine.md b/.claude/nexgen/harvest/02-prozentrang-shannon-doctrine.md new file mode 100644 index 000000000..8d702a46f --- /dev/null +++ b/.claude/nexgen/harvest/02-prozentrang-shannon-doctrine.md @@ -0,0 +1,26 @@ +# Reader 2 — Prozentrang / bucket / Shannon doctrine (verbatim, 2026-09-05) + +## (a) Prozentrang payload law + "bucket overflow → adjust thresholds" +Stated in `.claude/knowledge/observer-effect-tfpn-doctrine.md` §2 and mirrored in `.claude/plans/cycle-loop-closure-driver-v1.md:1481-1489` (§12.9a.1) and `.claude/board/exec-runs/d-blw-5-design-main-thread.md:71-75` (§c). +- The law: never inject the raw statistic (κ/φ/`BinaryAssociation`) — a scalar is trivially echoable and builds the anchoring/Goodhart fixed point into the instrument. Inject exactly two things: (1) the distribution SHAPE over the prior pool — a palette256/HDR-bucketed census (banded exposure, popcount-stacking early exit, CI thresholds, preheat + rolling-floor bucket, "the Belichtungsmesser reading"); (2) the Prozentrang — percentile rank of the observed association within that prior distribution. `shape₀ × rank₀` is ALL that is injected, frozen at V₀ (single-measurement law, §3). +- Machinery anchors cited (FINDING): `ndarray::hpc::cascade::expose(distance) -> Band` (`cascade.rs:162-175`) + `recalibrate(&mut self, alert: &ShiftAlert)` (`cascade.rs:211`); `ndarray::hpc::statistics::percentile(&self, p)` (`statistics.rs:41`). +- D-BLW-5 realization (`d-blw-5-design-main-thread.md:71-76`): 16-bucket histogram in Fisher-2z space (equal-width in 2z ≈ equal-information buckets), `rank₀` = Prozentrang within the pooled prior. +- "bucket overflow → adjust thresholds" — this exact framing does NOT appear anywhere in doctrine, plan, board, or code. Closest analog: recalibration on `ShiftAlert` and the phrase "preheating + rolling floor bucket". + +## (b) Shannon/entropy primitives in code +- `crates/thinking-engine/src/qualia.rs:688` — `fn shannon_entropy(energy: &[f32]) -> f32`, unnormalized; used normalized (`/ n.max(1.0).ln()`) at `:111`, `:159` +- `crates/lance-graph-contract/src/thought_atoms.rs:21` — normalized Shannon entropy of a weight vector in `[0,1]`, `H(p)/ln(n)`; `None` for empty, `Some(0.0)` degenerate +- `crates/lance-graph-cognitive/src/search/distribution.rs:157` — `pub fn entropy(&self) -> f32` — INT4 histogram (bits) +- `crates/lance-graph-cognitive/src/spectroscopy/features.rs:89` — `fn compute_entropy(pops: &[u32; CONTAINER_WORDS]) -> f32` — word-level popcount histogram, normalized by `log2(CONTAINER_WORDS)` +- `crates/lance-graph-planner/examples/entropy_surface_census.rs:238` — `f_shannon_entropy`; forms A–G +- `crates/lance-graph-planner/src/nars/insight.rs:132,178,183,258` — `truth_entropy` (10 bins) +- `crates/lance-graph/src/graph/arigraph/orchestrator.rs:335,383`, `sensorium.rs:29,155`, `contract/src/sensorium.rs:20`, `thinking-engine/src/osint_bridge.rs:25` — truth-entropy field docs/inline +- examples `hdr_audit.rs:302`, `tts_bgz_codebook.rs:212`, `calibrate_roles.rs:468`, `probe_token_bpe_geometry.rs:104`, `learning/src/cam_ops.rs:3038` — local computations +- `ndarray/src/hpc/styles/sdd.rs:2`, `cur.rs:2` — citations only + +## (c) Bucket rollover / Prozentrang scale adjust in code or doctrine +No such mechanism exists in code. All hits are negative doc-comments: +- `crates/lance-graph-contract/src/legacy_outliers.rs:27` — "lacking proper bucket rollover → a wide contiguous field with no HHTL" +- `crates/lance-graph-contract/src/identity_quad.rs:46` — "lacking proper bucket rollover — no. Each slot's capacity is the size of…" +- `crates/lance-graph-contract/examples/probe_wordnet_44_activation.rs:100,227,278` — "saturates silently" warning; line 278 discusses "Expanding to", nothing implemented +Doctrine-side adjacent mechanisms only: (i) `cascade::recalibrate`; (ii) the append-only remeasure-guard ledger keyed `(statistic-id, arm, cohort, metric, version)` that ERRORs on a second write to a sealed key. diff --git a/.claude/nexgen/harvest/03-proprioception-ewa-mexican-mask-surface.md b/.claude/nexgen/harvest/03-proprioception-ewa-mexican-mask-surface.md new file mode 100644 index 000000000..1485b91be --- /dev/null +++ b/.claude/nexgen/harvest/03-proprioception-ewa-mexican-mask-surface.md @@ -0,0 +1,31 @@ +# Reader 3 — proprioception, EWA/temporal sandwich, Mexican hat, 16k search, mask surface (verbatim, 2026-09-05) + +## `crates/lance-graph-contract/src/proprioception.rs` +- No `StateClassifier` trait in this file — module implements `ProprioceptionAxes`/`AnchorState`/`StateAnchor` +- 7 anchors + rungs (`ANCHOR_REGISTRY`, line 278-300+): `Intake` rung 5, `Focused` rung 4, `Rest` rung 3, `Flow` (coords `[0.7,0.5,0.3,0.6,0.9,0.6,0.7,0.2,0.8,0.7,0.7]`), `Observer`, `Balanced`, `Baseline` (enum line 161-176, `ALL` line 180-188) +- Nearest-anchor vs softmax: doc line 19-23 only; no concrete fn in range read +- Thresholds: `drive_ratio` cutoffs line 133-139 / 247-254 — `phi < 1.0` → `Explore`, `phi < 1.8` → `Exploit`, else `Reflect`. `STATE_DIMS = 11`, `CORE_AXES = 7`, `DRIVE_AXES = 4` + +## `ndarray/src/hpc/pillar/temporal_sandwich.rs` (Pillar-8) / `ewa_sandwich_2d.rs` (Pillar-6) +- temporal: SPD preservation of `Σ_{t+1} = M_t·Σ_t·M_tᵀ` across Cardiac/Respiratory/Micro; `sandwich_update_3x3` line 170; `is_spd_3x3` line 206; `PILLAR_8_PSD_THRESHOLD: f64 = 0.0` (line 59, placeholder); `SIGMA_CARDIAC=0.05`, `SIGMA_RESPIRATORY=0.20`, `SIGMA_MICRO=0.001`; `N_PATHS=1000`, `N_SUBSTEPS=30`; TODO line 57-64 `calibrate-pillar-8-σ_temporal` (comment says 0.10, const is `0.0`) +- ewa: `ewa_sandwich_step_2d` line 118; `PILLAR_6_PSD_THRESHOLD = 0.10` (line 53); `SIGMA_STEP = 0.2`; `N_PATHS=1000`, `N_HOPS=10`, `SPD_EPS=1e-9`; TODO line 54-58 `calibrate-pillar-6-σ_step` — lowered from 0.999 to 0.10 "denormal-tolerant placeholder", σ_step drives Σ to denormal in <30 hops + +## `ndarray/src/hpc/pillar/mexican_hat.rs` (Pillar-15) +- DEFERRED pending DoG kernel in `ndarray::hpc::dragonfly` +- κ band `σ_s/σ_c ∈ [1.5, 3.0]` (line 32-36); sweep `{1.1,1.3,1.5,1.8,2.0,2.5,3.0,4.0}` (line 54) +- Certifies (when active): DoG unimodality — single positive critical point, `DoG(0)>0`, decay to 0, exactly one root, second-derivative ratio budget +- `prove_pillar_15()` (line 111) returns placeholder `passed=true`, n_paths=0 +- Activation gate: public `dog_eval(r, sigma_c, sigma_s) -> f32` + +## `crates/holograph/src/width_16k/search.rs` +- Cascade diagram (line 7-26): L0 schema predicate → L1 Belichtungsmesser 7-point (comment only) → L2 block-masked StackedPopcount → L3 exact +- `passes_predicates` (line 251); `masked_distance` (line 393, no early exit); `masked_distance_with_threshold` (line 415) — per-block partial sums, `if total > threshold { return None }` (line 424-438) +- `search()` (line 447): predicate → thresholded distance → `partition_point` insert → tightens `current_threshold` to kth-best (line 483) +- `bloom_accelerated_search` (line 742), `rl_guided_search` (line 829) reuse the pattern +- No stride-sampling implementation in this file + +## `ndarray/src/simd_int_ops.rs` mask surface +- `eq_u32_to_mask(values: &[u32], needle: u32, out_words: &mut [u64])` — line 562 +- `eq_u32_strided_to_mask(...)` — line 629 +- `mask_and(a, b, dst)` — 775; `mask_or` — 807; `mask_and_assign(dst, src)` — 835; `mask_or_assign` — 861; `mask_andnot(a, b, dst)` — 904; `mask_andnot_assign(a, b)` — 932 +- `mask_ternlog(a, b, c, dst)` — 983; `mask_ternlog_assign(a, b, c)` — 1015 diff --git a/.claude/nexgen/harvest/04-sigker-power-kernels.md b/.claude/nexgen/harvest/04-sigker-power-kernels.md new file mode 100644 index 000000000..de9930b12 --- /dev/null +++ b/.claude/nexgen/harvest/04-sigker-power-kernels.md @@ -0,0 +1,10 @@ +# Reader 4 — sigker / power kernels (verbatim, 2026-09-05) + +- No "power kernel" concept exists in either workspace. Only unrelated power iteration: `ndarray/bf16_test_src/main.rs:1029` (PCA), `crates/lance-graph/src/graph/arigraph/ppr.rs:30` (PageRank), `crates/bgz-tensor/src/matryoshka.rs:304,396` (SVD), `crates/thinking-engine/QUICKSTART.md:135` (ReLU attractor collapse) +- ndarray side: `hpc/signature_pde.rs` (Goursat-PDE SIMD wavefront), `hpc/randomized_signature.rs`, `hpc/pillar/lattice_signature.rs`, `pillar/signature.rs` +- sigker crate: `lib.rs:1-79` (L52-58 Pillar 11 activation + W1.5 gate); `kernel.rs:37,68,129` — three kernels; `codec.rs:1-68`; `cubature.rs:54,83`; `log_signature.rs:44`; `randomized.rs:107`; `crates/jc/src/hambly_lyons.rs`, `jc/src/sig_checksum.rs` +- Plan: `.claude/knowledge/ndarray-vertical-simd-alien-magic.md:63-65,94-126`; `.claude/plans/pillar11-signature-certification-unification-v1.md` +- (a) Signature is over a discrete path `x: &[Vec]` (`kernel.rs:38`). Routes: `signature_kernel(x,y,depth)` truncated `K_N = Σ_{k=0..N}⟨S^k(X),S^k(Y)⟩` (line 37-54; tests depth 2/4/6/8); `signature_kernel_pde(x,y)` untruncated via `ndarray::hpc::signature_pde::signature_pde_sweep`, O(T₁·T₂·d) (line 68-127); `linear_path_kernel_closed_form(u,v)` = `I₀(2√⟨u,v⟩)` (line 129-151). W1.5 primitives: `F32x16::shuffle_product_lift(a,b,depth)` + `signature_pde_sweep`; `F32x16::random_proj_step(state, seed, depth)`; `I16x16::lyndon_pack(basis_idx)` / `lyndon_unpack_batch`. PDE half shipped; #7 randomized projection, #8 Lyndon pack remain (and #8 is not a SIMD primitive per E-A-SKETCH-THAT-MISSED-TWICE-…) +- (b) "power kernel": not a term anywhere +- (c) Pillar 11 gate: two conjunctive clauses — sigker benchmarked at production widths (PATH_DIM=4, PATH_LEN=64, N_PATHS=256) AND jc Pillar 11 active since PR #348; both TRUE 2026-09-02 → gate open. Two different "Pillar 11"s exist (ndarray `prove_pillar_11` on `signature_d2_deg3` vs jc Hambly-Lyons on `sigker::signature_truncated`) — name collision, unification plan unresolved +- (d) No link between sigker and masks/popcount/HDR anywhere. Only tie: `alien-magic.md:125` "the W1.5 sigker positioning is the architectural counterpart to I-NOISE-FLOOR-JIRAK (sigker bypasses what Jirak quantifies)" diff --git a/.claude/nexgen/harvest/10-pr-sweep-lance-graph-1175-1166.md b/.claude/nexgen/harvest/10-pr-sweep-lance-graph-1175-1166.md new file mode 100644 index 000000000..fd75f9fc7 --- /dev/null +++ b/.claude/nexgen/harvest/10-pr-sweep-lance-graph-1175-1166.md @@ -0,0 +1,12 @@ +# PR sweep — lance-graph #1175–#1166 (verbatim, 2026-09-05) + +- #1175 — regrade membrane-tiers ledger L1/L2 CLOSED. Markdown only. SYNERGY: NONE. +- #1174 — membrane-tier doctrine + `kernel-membrane-warden`/`bbb-warden`. SYNERGY: WEAK — names `ternlog`/`mask_*`/`AND3` as the T1 vocabulary any mask-cache work must route through. Ruling: "T2 may only speak T1's NAMES... never spell a T1 op out of smaller ops." +- #1173 — untracks 27 `Cargo.lock`. NONE. +- #1172 — sigker clippy fixes + CI lint. WEAK. +- #1171 — epiphany: board gates shipped with the defect they catch. NONE. +- #1170 — three review fixes on board gates. NONE. +- #1169 — retroactive hygiene for #1167/#1168. NONE. +- #1168 — no-new-decay gate for `file:NNN` citations. NONE. +- #1167 — append-only files never shrink gate. NONE. +- #1166 — `sigker::RandomizedSignatureBuilder::encode` → `ndarray::hpc::randomized_signature::randomized_signature_sweep` (F64x8); ragged-path validation; lane-type doc corrections (`E-A-SKETCH-THAT-MISSED-TWICE-WILL-MISS-A-THIRD-TIME-1`). STRONG. Note: "the doc's '7-13× compression' headline is asymptotic... ~2.1× at d=4,N=2". diff --git a/.claude/nexgen/harvest/11-pr-sweep-lance-graph-1165-1156.md b/.claude/nexgen/harvest/11-pr-sweep-lance-graph-1165-1156.md new file mode 100644 index 000000000..6cbe8f62d --- /dev/null +++ b/.claude/nexgen/harvest/11-pr-sweep-lance-graph-1165-1156.md @@ -0,0 +1,12 @@ +# PR sweep — lance-graph #1165–#1156 (verbatim, 2026-09-05) + +- #1165 — board correction of #1163 verdict. NONE. +- #1164 — (closed unmerged) four superseded readings. NONE. +- #1163 — 58.2% inversion + stranded-branch method fix. WEAK: `atlas.rs:465` agreement between two DAG-depth ancestor mechanisms. +- #1162 — (closed unmerged) Storno: HHTL 58.2% inversion, 352/2,048 non-zero key bytes in `all-lanes.soa`, rail-head agreement 314/314. WEAK. +- #1161 — `plan_dids.py` CI gate. NONE. +- #1160 — D-DCR-4 Σ-transport gate: `jc::ewa_sandwich` vs `lance_graph_contract::sigma_propagation::ewa_sandwich` byte-identity (`Spd2`, `0.5*(r01+r10)` symmetrization). STRONG. Ruling: "lance-graph-contract is zero-dependency by design; its manifest forbids even optional path deps" → forced-copy + gate pattern. +- #1159 — regrade dangling refs to retracted `epistemic_bassin`/`basin_lanes`/loco band 0x87..0x8B. WEAK landmine: retracted design had `sweep_ternlog`/`eval_ternlog`, `ASKED_CONTESTED 0x80`…`ASKED_SILENT 0x02`. Ruling `E-SIX-SEMANTIC-FAMILIES-MUST-NOT-IMPERSONATE-EACH-OTHER-1` — only `TERNLOG = FnIndex(0x86)` survives. +- #1158 — D-PRLR-1..5 for R2IL regfile probe. NONE. +- #1157 — R2IL session coordination; TERNLOG-as-address vs palette-vocabulary. WEAK: reaffirms `ogar_loco::TERNLOG = FnIndex(0x86)`, one FnIndex / 8-bit truth table addressing 256 combinators; minted-only, unconsumed. +- #1156 — untracked-plan census 53/208. NONE. diff --git a/.claude/nexgen/harvest/12-pr-sweep-lance-graph-1155-1146.md b/.claude/nexgen/harvest/12-pr-sweep-lance-graph-1155-1146.md new file mode 100644 index 000000000..2a3fa1062 --- /dev/null +++ b/.claude/nexgen/harvest/12-pr-sweep-lance-graph-1155-1146.md @@ -0,0 +1,12 @@ +# PR sweep — lance-graph #1155–#1146 (verbatim, 2026-09-05) + +- #1155 — D-R2IL-1..5 backfill. WEAK: §4 "12 bytes a dumb ClassView-projected register; macro vocabulary is a PALETTE not a `FnIndex`". +- #1154 — `confidence_entropy` routed through `thought_atoms::normalized_entropy`; zero-mass convention pinned by `an_empty_arena_has_zero_truth_entropy_not_one`. STRONG. Cross-ref ruling: "Do not rewrite good SIMD in R2IL to claim purity". +- #1153 — entropy-surface census: 7 implementations, 4 conventions, consolidation target `thought_atoms` not `jc`; forms A–G, C1/C2/C3 falsifiers. STRONG. Ruling (operator 2026-08-31): "shannon, Mengenlehre etc sind universale denk atome" → entropy home = `contract::thought_atoms`. +- #1152 — `AlphaTunnel`: ten LANES over ONE `AlphaAllocation`, read/write split. WEAK. Ruling `F-RLR-12`: "A correction must cite the type's definition, not re-describe its shape." +- #1151 — D-TEH-4 dtype collapse behind `BuiltEngine`, `dtype_parity.rs`. WEAK. +- #1150 — `sigker::log_signature` wired (was dead code), Lyndon projection bug fixed. WEAK. +- #1149 — board close (Pillar-11 Cholesky shipped in ndarray #291). NONE. +- #1148 — hygiene for #1145. WEAK: `witness_fabric::elect_and_bind`/`WitnessLens::bind_election` "order-independent + idempotent". +- #1147 — `sigker::signature_kernel_pde` → `ndarray::hpc::signature_pde::signature_pde_sweep` (F64x8, AVX-512/AVX2/NEON), W1.5 gate #6 closed. STRONG. Ruling: "ndarray is mandatory everywhere numeric/computational code runs". +- #1146 — hygiene for #1144. NONE. diff --git a/.claude/nexgen/harvest/13-pr-sweep-lance-graph-1145-1136.md b/.claude/nexgen/harvest/13-pr-sweep-lance-graph-1145-1136.md new file mode 100644 index 000000000..a24e34e4b --- /dev/null +++ b/.claude/nexgen/harvest/13-pr-sweep-lance-graph-1145-1136.md @@ -0,0 +1,13 @@ +# PR sweep — lance-graph #1145–#1136 (verbatim, 2026-09-05) + +- #1145 — D-POP-2 `elect_and_bind`/`WitnessLens::bind_election` writes elected Quorum/Contradiction loci into the row's own CausalWitness register. STRONG. Ruling: "the fabric never reads what it computes". +- #1144 — D-TEH-3 fate probes: `semantic_chunker`, `spiral_segment` both KILL, stay LAB; 25 clippy fixes. WEAK. +- #1143 — calibration math → `jc::{drift, quorum, reliability}` (`pairwise_agreement_u8`/`QuorumLevel`, `reencode_drift`); lab `cronbach.rs` deleted. STRONG. Ruling `E-JC-IS-THE-HOME-OF-ALL-CALIBRATED-MATH-1`. +- #1142 — `nars::ghost_prior::GhostPrior` in planner; floor gate default `Marker` (0.1 clamp) over `Trace` (0.001 prune). STRONG. Ruling: "Lingering trace ≠ the −6 counterfactual lane". +- #1141 — PROBE-HOUSE-DIFFERENTIAL-1; variant 1 KILL, variant 2 base PASS. WEAK. Ruling: "The periphery of a stratum is the other strata". +- #1140 — Pillar-11 closure: guard correction (arXiv-e → Annals-2e), ndarray SIMD substrate debt, A0/A1/A2 Goursat SIMD bench (F64x8 FMA), AMX/SPR-vs-EMR note. STRONG. Ruling (TD-PILLAR11): "call the named algorithm; if absent, compose from ndarray::simd::{F64x8,F32x16,I16x32}::*; never a consumer-local arithmetic path". +- #1139 — `bridge_gate` moved to contract; callcenter drops thinking-engine dep. NONE. +- #1138 — four operator rulings recorded. WEAK. "jc is the home of all scientifically calibrated math". +- #1137 — plans only. NONE. +- #1136 — PROBE-POP-READOUT-1 KILL (p@10 0.289→0.011); `curiosity_gestalt` rank-inert (ρ=1.000000 vs magnitude); frequency beats all cognitive arms. STRONG. Ruling: "any future frontier-ranking claim clears the frequency control first". +- Cross-cutting: none of the ten mention ternlogq, u64 row masks, version-keyed slab cache, Mexican hat, Shannon, proprioception, EWA/SPD by name. diff --git a/.claude/nexgen/harvest/14-pr-sweep-lance-graph-1135-1126.md b/.claude/nexgen/harvest/14-pr-sweep-lance-graph-1135-1126.md new file mode 100644 index 000000000..b8fda54ae --- /dev/null +++ b/.claude/nexgen/harvest/14-pr-sweep-lance-graph-1135-1126.md @@ -0,0 +1,12 @@ +# PR sweep — lance-graph #1135–#1126 (verbatim, 2026-09-05) + +- #1135 — buildup survey v1: 35 operators/atoms, 6 families; inventories `helix::fisher_z`, `RollingFloor::{quantize,occupancy,drift_score}`, `Cam96Space::distance`, `FrontierEdge::curiosity`, `shuffle_beliefs_null`. STRONG. Ruling: "Population-derived semantic geometry is an accepted vacancy". +- #1134 — removal of `basin_lanes.rs`/`epistemic_bassin.rs` ("September recovery"). WEAK, history: deleted code briefly had Belnap masks + ternlog sweep. Ruling: "No replacement population-basin representation... an accepted vacancy." +- #1133 — literature harvest (NARS/bilattice/EIG/Hambly-Lyons/SPD-EWA) + jc Pillar 11 green for lattice walks + register audit. STRONG. Rulings: "MINT NOW is empty"; "a single `u8:u8` rail read as one axis is out of regime". +- #1132 — 24-axis basis v3 mirror (`epistemic_bassin::axes`), 6×4, mirrors OGAR 0x0334; witness group PROVENANCE/REVISION/QUORUM/CONTRADICTION. STRONG. Rulings: "no Hambly-Lyons axis while jc Pillar 11 is red"; "no domain FnIndex mints". +- #1131 — rung-per-classid ruling rescued; 256:256-by-classid verified. STRONG. Ruling: "one classid per rung level... vocabulary IS epistemic altitude... no rung classid is minted before the [rung 5-9] table." +- #1130 — loco band mirror `EPISTEMIC_CALLS`: TERNLOG/BELNAP_JOIN/INFO_GAIN/SIGMA_TENSION/ACCUMULATE/STANCE_ENTROPY; 256:256 ruling text. STRONG. Ruling: "the classid swaps the whole 256-entry palette (256:256)... core bytes are readable in every vocabulary by construction." +- #1129 — `EpistemicBassin24` pair `agree_u4[24]+disagree_u4[24]`; `info_gain_u4`, `sigma_tension_u4` (quarters of `pillar_5plus_bound`), `sweep_ternlog`/`eval_ternlog`. STRONG. Ruling: "`+3 + −3 = 0`... agreement/disagreement IS the interesting information"; "no lane" for Hambly-Lyons while Pillar 11 red. +- #1128 — `BasinLanes` (G24N4), `accumulate_children` (exact-sum-then-clamp), `hhtl::{missing_ancestors,direct_children}`. WEAK/STRONG-adjacent. Ruling: "mechanical hydration mints structure... epistemic output = silence". +- #1127 — value-lane census; `G24N4` already ships on `CausalWitnessFacet`. WEAK. Ruling: "`G24N4` is a lane shape name, never a `CascadeShape` variant". +- #1126 — 29 TSV lanes appended; palette FULL 256/256 const-asserted. WEAK. Ruling: "no append margin remains". diff --git a/.claude/nexgen/harvest/15-pr-sweep-ogar-298-274.md b/.claude/nexgen/harvest/15-pr-sweep-ogar-298-274.md new file mode 100644 index 000000000..6889a7520 --- /dev/null +++ b/.claude/nexgen/harvest/15-pr-sweep-ogar-298-274.md @@ -0,0 +1,22 @@ +# PR sweep — OGAR #298–#274 (verbatim, 2026-09-05) + +- #298 — retract epistemic band; drop ogar-epistemic. STRONG. Ruling: "the pair-specific epistemic band... aliased three distinct semantic families — episodic/Markov loci, qualia magnitude, and population-derived semantic geometry — into one type". Do NOT resurrect 0x87–0x8B. +- #297 — ogar-epistemic 24-axis (0x0334). NONE (retracted). +- #296 — mint 0x86..0x8B. STRONG but retracted except TERNLOG(0x86): "a generic 3-input mask combinator whose value byte is the truth table... no named table set is canonical"; batch SIMD = ndarray `ternlog`. +- #295 — `BasinCodebook`/`BasinCodebookBuilder` (`seal()` consumes builder, no `&mut` after mint), `ConceptDomain::resolve_operand`. STRONG: build-once/freeze precedent for a version-keyed immutable slab. +- #294 — `row_schema_of` dual-address fix. WEAK. +- #293 — OBO producer flip + `examples/rekey_domain.rs` in-place classid rewrite of 512-byte rows. WEAK. +- #292 — mint 0x90..0x9A. NONE. #291 — revert 0x03XX rows. NONE. #290 — take back 0x03XX double-mint. NONE. +- #289 — R2IL PUSHES→segmentation; D-CFW-LADDER byte⊇exact-area⊇register⊇increment; `statement_bounds` (R5 maskable unit); blind spots at coarser granularity. STRONG. +- #288 — `CallMask` `Box<[u64]>`→inline `[u64;3]`, `Copy`, lazy `set_indices()` (word scan + `trailing_zeros` + `w &= w-1`), tail-mask fix in `not()`. STRONG. +- #287 — docs breadcrumb. NONE. +- #286 — 0xC6 classids + execute-never-convert ruling. WEAK. +- #285 — R2IL as loco Vocabulary + `project(&slab, shape, &mask)` lazy masked iteration; "Reshuffling is a re-READ, never a re-WRITE". STRONG. +- #284 — 0xC6 Mmio. NONE. #283 — DocIr harvest (draft). NONE. +- #282 — typed_field + W4 ActionDefs; `capability_registry::domain_tables()`. WEAK. +- #281 — PROBE-LOCO-INTERPRETER-1: `VAR_SET`/`VAR_CHANGE` push with no DROP/POP → `statement_bounds` refuses imperative sequences (`DanglingOperands`). WEAK caveat. +- #280 — debug info off. NONE. +- #279 — dismech search band 0xA3..0xA9; `residue_band` (cardinality → 3-bit band), pothole→rung degradation (Surface/Association/Relation/Counterfactual), `Throttle::hub_indegree`, `diffuse_floor` unset ("no measured value exists"); "it is not evidence... search-economic". STRONG. +- #278 — askama_axum. NONE. #277 — 0xC0 Panama alone. NONE. #276 — C-band reserve. NONE. +- #275 — 19 DisMech predicates. WEAK. #274 — 0x0333 reservation. NONE. +- Top hooks named by the reader: #288+#285 (CallMask + project), #295 (BasinCodebook seal), #289 (visibility ladder), #279 (residue_band/diffuse_floor), #296/#298 (TERNLOG 0x86 only). diff --git a/.claude/nexgen/harvest/16-pr-sweep-ndarray-301-277.md b/.claude/nexgen/harvest/16-pr-sweep-ndarray-301-277.md new file mode 100644 index 000000000..d0a8ef2be --- /dev/null +++ b/.claude/nexgen/harvest/16-pr-sweep-ndarray-301-277.md @@ -0,0 +1,23 @@ +# PR sweep — ndarray #301–#277 (verbatim, 2026-09-05) + +- Rulings quoted: "VPABSB does NOT saturate `i8::MIN`"; W1a struct-method litmus (`vertical-simd-consumer-contract.md:325-326`), with #280/#301 deviating to free-fn family shape by recorded justification; "No `&mut self` during computation. Ever." (data-flow.md) +- #301 — `mask_ternlog`/`mask_ternlog_assign` + stride-4 fast path in `eq_u32_strided_to_mask`. STRONG. +- #300 — stop committing Cargo.lock. NONE. +- #299 — `array_chunks` into `heel_f64x8::cosine_f32_to_f64_simd`. WEAK. +- #298 — docs. NONE. #297 — `BlasLevel3` re-export. NONE. #296 — W1.5 docs backfill. WEAK. #295 — docstrings. NONE. +- #294 — `randomized_signature_sweep` (Cuchiero, F64x8 GEMV+axpy). NONE for masks. +- #293 — `signature_pde_sweep` Goursat wavefront F64x8. NONE for masks. +- #292 — `pillar::lattice_signature` bit-exact i128 + Hambly-Lyons Thm 5/6; "scalar integer reference lane on purpose… `ndarray::simd` has no i128 lane". NONE. +- #291 — Pillar 11 PSD gate → jittered Cholesky (`gram_is_psd`, `PSD_JITTER = 1e-4`, `hpc::lapack::LapackOps::cholesky`), bit-exact CI pins. STRONG. +- #290 — signer-parity crate, PSD at depth-∞, hand-rolled Cholesky. WEAK. #289 — docs. NONE. #288 — drop burn submodule. NONE. +- #287 — scalar `U64x8::ternlog` IMM const-guard fixed; AVX-512 verification claimed→measured. STRONG. +- #286 — blackboard record for #285; deviation: "the pair follows the free-fn family shape rather than the struct-method litmus… council-surfaced (5+3, S2-7)". STRONG by reference. +- #285 — `U64x8`/`U32x16` `andnot` + `ternlog::` on all six backends + `simd::ternlog::{AND3,OR3,MAJ3,…}`; consumer wired to lance-graph `FieldMask`/`WideFieldMask`. STRONG. +- #284 — wasm32 docs. NONE. +- #283 — `masked_strided_group_sum` (scalar by design, memory-bound stride; i128 accumulate, overflow-checked into i64). STRONG. +- #282 — debug off. NONE. #281 — chacha20 spec. NONE. +- #280 — `mask_andnot`/`mask_andnot_assign`; "a lone struct-method member sitting beside four free-fn siblings would fragment… the polyfill surface." STRONG. +- #279 — `eq_u32_strided_to_mask`, `mask_and`/`mask_or`(+assign), `masked_sum_i32`, `MultiLaneColumn::iter_u32x16`. STRONG. +- #278 — wasm32 SIMD128 guide. NONE. +- #277 — `ClamTree::build_with_distance`, `TreeDistance::{Ptr,Dyn}`, `is_metric()` carried not assumed, `rho_nn` hook. STRONG. +- Not touched in this range: Prozentrang/percentile, Mexican hat, Shannon, EWA/temporal pillars, `hpc::cascade`. diff --git a/.claude/nexgen/plans/nexgen-mask-histogram-thresholds-v1.md b/.claude/nexgen/plans/nexgen-mask-histogram-thresholds-v1.md new file mode 100644 index 000000000..67beeb126 --- /dev/null +++ b/.claude/nexgen/plans/nexgen-mask-histogram-thresholds-v1.md @@ -0,0 +1,180 @@ +# nexgen-mask-histogram-thresholds-v1 — the exposure meter is a nested mask set + +**Status:** PROPOSAL (2026-09-05). Nothing below is built or measured. +**D-ids:** D-NXG-1 … D-NXG-12 (rows on `.claude/board/STATUS_BOARD.md`). +**Evidence:** `.claude/nexgen/harvest/` (11 verbatim agent reports, 2026-09-05). +**Board block:** `.claude/board/EPIPHANIES.md` 2026-09-05 `E-NXG-*` (16 entries). +**Location note:** this plan lives under `.claude/nexgen/plans/`, not +`.claude/plans/`. `supersession_index.py` and `plan_dids.py` scan only +`.claude/plans/*.md`, so the D-ids here are visible to the STATUS_BOARD and +invisible to the index's coverage column. That is a known gap, recorded on +purpose: moving the file is a one-line change once the folder earns a second +plan. Do not read the coverage column's silence as "untracked". + +--- + +## 0. The one sentence + +A Belichtungsmesser reading over N rows is a chain of nested row masks +`M_1 ⊆ M_2 ⊆ … ⊆ M_16`; its histogram is 16 popcounts; a row's Prozentrang is +the index of the innermost mask containing it; bucket i is `M_i ∧ ¬M_{i−1}` +(`vpternlogq` immediate `AND_ANDNOT2 = 0x10`); bucket overflow is a popcount +test; rollover is a mask split keyed by Lance version. Threshold, band, bucket, +rank, cache entry and basin are the same object read five ways. + +## 1. What exists (from the harvest, file:line as of 2026-09-05) + +| mechanism | where | shape today | gap | +|---|---|---|---| +| reject floor | `ndarray/src/hpc/cascade.rs:137` | `mu + 3σ`, k fixed | Gaussian-tail assumption; Jirak note at `rolling_floor.rs:25` says it is wrong | +| bands | `cascade.rs:162`, `rolling_floor.rs:140` | quarters of t | derived from t only, no rank | +| preheat | `rolling_floor.rs:93,230` | copy mu/σ/n coarse→fine | two scalars where a survivor set exists | +| early exit | `rolling_floor.rs:239`, `width_16k/search.rs:483` | first Alarm / kth-best tightening | two rules for one idea | +| buckets | `hdr_cascade.rs:200` (`calculate_sweet_spot`), `:625` (`classify_signal`), OGAR `residue_band` | hand `match` tables | no overflow handling, no rollover | +| Prozentrang | doctrine §2, D-BLW-5 design | 16-bucket Fisher-2z histogram | **unbuilt**; `statistics::percentile` is a batch sort | +| bucket rollover | `legacy_outliers.rs:27`, `identity_quad.rs:46` | warnings only | **absent everywhere** | +| Mexican hat | `hdr_cascade.rs:128` | piecewise-linear ramp | Pillar-15 DEFERRED, placeholder `passed=true` | +| entropy | `thought_atoms::normalized_entropy` (ruled home, #1153/#1154); popcount-histogram form at `spectroscopy/features.rs:89` | scattered, consolidating | not wired to thresholds | +| EWA sandwich | `contract::sigma_propagation::ewa_sandwich` ≡ `jc::ewa_sandwich` (#1160) | certified byte-identical | `PILLAR_6_PSD_THRESHOLD` = 0.10 placeholder; σ_step=0.2 denormals in <30 hops | +| mask primitives | `ndarray/src/simd_int_ops.rs:562-1015` | `eq_*_to_mask`, `mask_{and,or,andnot,ternlog}[_assign]` | complete for this plan | +| ownership shape | OGAR `BasinCodebookBuilder::seal()` (#295) | build-freeze-no-`&mut` | reusable as-is | +| iteration shape | OGAR `CallMask` inline `[u64;3]` + `set_indices()` (#288) | word scan + `trailing_zeros` | reusable as-is | +| surviving combinator | `ogar_loco::TERNLOG = FnIndex(0x86)` (#296/#298) | value byte = truth table | 0x87–0x8B RETRACTED, never resurrect | + +## 2. Architecture entropy — the T0..T3 shape + +The membrane doctrine (`.claude/knowledge/membrane-tiers.md`) says a tier may +only know the vocabulary of the membrane beneath it, crossing by NAME. Applied +here, the histogram collapses vocabulary at every tier. "Entropy" below is the +count of distinct concepts a tier must name; the plan's goal is to lower it +at every membrane without losing a reading. + +### T0 — substrate (bytes, lanes) +- Owns: `u64` words, `vpternlogq`, popcount, the Lance version stamp. +- Knows nothing of bands, ranks, or thresholds. +- Entropy before: same. After: same. T0 is already minimal; nothing lands here. + +### T1 — primitive (`ndarray::simd`, `lgj-abi/kernels.rs`) +- Owns today: `mask_*`, `eq_*_to_mask`, `ternlog::{AND3,…}`, `masked_strided_group_sum`. +- Adds (D-NXG-2): `mask_bucket::(m_i, m_prev, dst)` is NOT a new + primitive; it is `mask_ternlog::` by name. The only genuinely new + T1 op is `popcount_words(&[u64]) -> u64` if it is not already exposed as a + named primitive (harvest did not find one under that name; verify before + minting). Everything else composes. +- Adds (D-NXG-3): a `bisect_column_by_mask(col: &[u16], within: &[u64], target_popcount) -> boundary` + partial-popcount bisection. This is the one primitive that reads a value + column, and it must stay a T1 NAME so T2 never spells it out of a sort. +- Entropy: +1 or +2 names. Every T2 concept below is expressible in the existing + eight immediates plus these. + +### T2 — selection (`where`/`hop`/`plan_eval`, the slab cache) +- Owns: `NestedBands { boundaries: [u16; B], masks: [MaskHandle; B], popcounts: [u64; B], version }`, + sealed per `BasinCodebookBuilder::seal()` shape (D-NXG-1). +- Replaces, by name, four T2 concepts with one: + `Cascade::Band`, `FloorBand`, `QualityTracker` buckets, `residue_band` are + all `bucket_index(row) = partition_point(masks, row)`. +- Rank (D-NXG-4): `prozentrang(row) = bucket_index(row)` scaled to `[0, B)`. + Shape = `popcounts`. This IS the doctrine's `shape × rank`, computed, not + designed. +- Rollover (D-NXG-5): when `popcounts[i] − popcounts[i−1] > budget`, call + `bisect_column_by_mask` inside bucket i, insert the boundary, mint the new + mask under the current version. Old masks are never rewritten; the old + `NestedBands` stays addressable by its version. +- Reject floor (D-NXG-6): `boundary[B−1]` at a chosen rank, not `mu + kσ`. σ + becomes `RollingFloor::z()` diagnostic output only. +- Preheat (D-NXG-7): `M_fine_domain = M_coarse[weak]`; the fine tier's initial + `NestedBands` is the coarse one restricted by AND. +- Early exit (D-NXG-8): one rule, `popcount(survivors) ≤ k_wanted`, replacing + first-Alarm and kth-best tightening. +- Entropy: T2 drops from ~6 threshold vocabularies to 1 (`NestedBands`) plus + the rank read. The T1/T2 warden's NAMED test passes by construction: every + op above is a `mask_ternlog` immediate or one of the two T1 names. + +### T3 — intent (Java facade, R2IL, planner, consumers) +- Sees: `Rank(u8)`, `Shape([u64; B])`, `Version`. Never a boundary value, never + a σ, never a mask word (BBB rule: names cross, byte positions do not). +- `WideFieldMask`/`FieldMask` are untouched (operator ruling 2026-09-04: no + removal, no demotion without permission). The histogram is additive beside + them. +- Entropy: T3 gains one name (`Rank`) and loses every threshold constant it + currently pins (`threshold_l0/l1/l2`, `DEFAULT_EXCITE/INHIBIT`, `k`). Those + become T2-internal, learned from the slab. + +### R2IL / low-code +- `where rank(col) <= 3` is the whole surface. The compiler lowers it to a + bucket-index compare, which is one AND against a cached mask. This is the + "Java becomes low-code" consequence the membrane doctrine promised, arriving + through thresholds rather than through field masks. + +## 3. Folding the epiphanies — what falls out that nobody asked for + +Each row: the epiphany it folds from → the unforeseen consequence → the +falsifier that would kill it. Ordered by how many rooms ahead it sits. + +| rooms | folds from | what falls out | falsifier | +|---|---|---|---| +| 1 | E-NXG-1 + E-NXG-3 | **The 16-bucket histogram is a 16-bit rank code per row.** Store `bucket_index` as a u8 column when the masks are cold; the masks regenerate from it by `eq_u8_to_mask` in one sweep. Cold storage = 64 KB for 65 536 rows, not 128 KB. | regenerate masks from the u8 column and assert bit-equality with the sealed masks | +| 2 | E-NXG-9 | **Histogram entropy is a self-timer for rollover.** `H(popcounts) → 0` triggers bisection before any bucket "overflows" numerically. Rollover becomes entropy-driven, not budget-driven. | on a bimodal distance column, entropy-triggered rollover must split earlier than budget-triggered | +| 3 | E-NXG-4 + E-NXG-11 | **σ is recoverable from the histogram for free** (variance of bucket centres weighted by popcounts). The EWA sandwich then propagates a Σ that was never stored, only read. `RollingFloor` loses its `mu/sigma` fields. | recovered σ vs Welford σ within Jirak-rate tolerance on real facet columns | +| 4 | E-NXG-7 + E-NXG-8 | **The eight named immediates are a complete cognitive ISA over masks.** AND3 narrow, AND_ANDNOT2 bucket/known-false, MAJ3 quorum, XOR3 disagreement, OR2_AND gated union. A "thinking style" at the mask level is a sequence of immediates, i.e. a byte string, i.e. a `Vocabulary` entry. This is `TERNLOG(0x86)` with its value byte read as a program. | any style in `contract::thinking` that cannot be expressed as ≤ 8 immediates over ≤ 3 masks | +| 5 | E-NXG-12 | **Proprioception is a histogram of histograms.** Seven anchors × 16 buckets = a 7×16 popcount matrix per window. State = the row with maximum mass; drive_ratio thresholds = two boundaries in that row's `NestedBands`. The 11-dim vector and nearest-anchor distance disappear. | a window where the vector classifier and the histogram classifier disagree, resolved by which one the operator-ruled anchor set calls correct | +| 6 | E-NXG-6 + E-NXG-2 | **Top-k and alarm are the same query.** Search asks "survivors ≤ k"; monitoring asks "survivors ≤ floor". The rolling floor IS a top-k with k = floor. `perturbation-sim`'s early-warning and `holograph`'s search share one T2 function. | one `NestedBands` driving both `stack_early_exit` and `width_16k::search` with identical exits on a shared fixture | +| 7 | E-NXG-14 | **The narrowing trajectory is a path; sigker compares trajectories.** Popcounts per tier = a length-4 path in ℝ⁴. `signature_kernel_pde` on two such paths is an order-sensitive "did these two queries think alike" scalar. Cheap enough (T=4) to run per query pair. Feeds the prefetch planner (E-NXG-*, prefetch frontier): prefer frontier masks whose trajectory signature is close to a hot one. | a pair of queries with identical final survivor sets but different tier trajectories must score < 1.0; identical trajectories = 1.0 | +| 8 | E-NXG-10 | **The Mexican hat is two boundaries; κ is a rank ratio; Pillar-15 certifies on the histogram.** DoG unimodality on a discrete 16-bucket histogram is a finite check (one local max, one annular min) that runs in the T2 seal, not a deferred continuous pillar. Pillar-15 can activate without `dog_eval`. | the seal must refuse a `NestedBands` whose excite/inhibit ratio leaves κ ∈ [1.5, 3.0] | +| 9 | E-NXG-13 | **A basin is a band, so the zero ladder is a missing mask.** `resolve(0) → None` becomes "no mask at this key"; the slab cache and OGAR's `BasinCodebook` are the same sealed object under two names. One `seal()`; one owner; version-keyed. | a basin lookup and a band lookup for the same `(classid, version, idx)` return the same handle | +| 10 | E-NXG-1 + version keying | **Time travel is free.** Because every `NestedBands` is version-keyed and never rewritten, "what was the rank of row r at version v" is a lookup, not a recomputation. The Prozentrang gains a temporal axis; the payload law's "frozen at V₀" is literally the key. | rank at V₀ read after 10 rollovers equals the rank sealed at V₀ | +| 11 | E-NXG-8 XOR3 | **Cross-version surprise is one instruction.** `M_v ⊕ M_{v−1}` per band = which rows changed rank between versions; its popcount is a drift meter with no σ. `ShiftAlert` becomes `popcount(xor) > budget`. | a version pair with identical masks yields zero drift; a rewritten column yields drift proportional to rows moved | +| 12 | rooms 4 + 11 | **Learning is a diff of programs.** If a style is a byte string of immediates (room 4) and version drift is a mask XOR (room 11), then "which style reduced drift most" is a scan over ≤ 256 byte strings against one XOR mask. NARS-style self-reinforcement without a model: keep the program whose survivors moved least. | two styles applied to the same version pair; the one with lower XOR popcount must be the one the existing NARS revision would also prefer, on the fixtures where NARS has a verdict | +| 13 | E-NXG-3 rollover | **Buckets can merge, not only split.** When two adjacent buckets both fall under budget/entropy, drop the boundary between them; the merged mask is `M_i ∨ M_{i+1}` (already nested, so just `M_{i+1}`). The histogram breathes with the distribution. B is not fixed at 16; it is bounded by the L2 budget. | on a collapsing distribution, B must fall; on a widening one, rise; never exceed the slab budget | +| 14 | E-NXG-5 + room 1 | **Preheat across mailboxes is a u8 column copy.** Because the cold form is a rank column, a sibling mailbox inherits a histogram by copying 64 KB and re-minting masks under its own version. No shared mutable slab, no baton, one-writer preserved. | two mailboxes seeded from the same rank column produce identical `NestedBands` under different versions | +| 15 | rooms 5 + 12 | **Proprioceptive drift is stylistic drift.** If state is a histogram row and style is an immediate program, then "the agent's state changed" and "the agent's thinking changed" are XORs on the same substrate. The MUL layer's homeostasis gate reads one popcount. | a held-constant style over a moving state must show state drift with zero style drift, and vice versa | +| 16 | E-NXG-2 + CallMask | **`set_indices()` over a bucket mask is a rank-ordered iterator.** Walking `M_i ∧ ¬M_{i−1}` for i = 0.. yields rows in Prozentrang order without a sort. Top-k is the first k yields. `partition_point` insertion disappears. | iteration order equals stable sort order by distance, within-bucket order unspecified and asserted so | +| 17 | rooms 10 + 16 | **A time-ordered rank walk is episodic recall.** Iterate buckets at version v, then v+1: the rows that appear earlier in the later walk are "what came into focus". This is the ±5 window of the Markov trajectory expressed as two mask walks. | on a synthetic stream where focus is known, the walk-diff must recover it | +| 18 | E-NXG-7 retraction | **Named epistemic calls come back as *derived* reads, never mints.** INFO_GAIN = `H(shape_v) − H(shape_{v−1})`; SIGMA_TENSION = the sandwich-propagated Σ (room 3); STANCE_ENTROPY = `H` of the 7×16 proprioception matrix (room 5). All three exist without a FnIndex, satisfying #298's family-separation ruling because each reads exactly one family's masks. | any derived read that has to AND masks from two semantic families is a violation and must be rejected at the seal | +| 19 | membrane doctrine | **`where` compiles to a rank compare; the Java facade never sees a threshold again.** Every `threshold_l0/l1/l2`, `k`, `DEFAULT_EXCITE` constant currently pinned above T2 is deleted from T3 signatures (additively: new signatures beside old, per the no-removal ruling). | `ApiSurfaceTest`-style reflective fence: no public T3 signature carries a numeric threshold parameter | +| 20 | rooms 4 + 19 | **R2IL gains a `RANK` macro, not a threshold macro.** The R2IL vocabulary (#285, "execute-never-convert") lowers `rank(col) <= n` to `CallMask` AND; the interpreter never sees a distance. | a lifted R2IL program that compares a distance directly is rejected by the vocabulary | +| 21 | room 13 + palette FULL | **Bands are the palette's growth path.** #1126 says the 256-entry palette is FULL. A `NestedBands` per classid is 16 entries per classid living in the slab, not in the palette; the 256:256-by-classid ruling (#1130) already says the classid swaps the whole palette. Band index = a per-classid sub-palette that costs no palette slots. | zero new palette lanes minted by this plan, const-asserted | +| 22 | room 2 + room 11 | **Entropy of the XOR is a novelty meter.** `H(popcounts(M_v ⊕ M_{v−1}))` is high when change is spread across ranks, low when concentrated. Concentrated change = one basin moved = an event; spread change = noise or drift. This is the Belichtungsmesser's `adaptive_resolution(query_entropy, corpus_cv)` match table replaced by two numbers. | on injected single-basin change, XOR entropy must fall; on uniform noise, rise | +| 23 | E-NXG-16 | **Correlation between bands is measurable from the slab.** `popcount(M_a ∧ M_b) / min(popcount)` for two predicates is their overlap; the prefetch planner's independence assumption is checked, not assumed, by one AND. Store the overlap matrix (B×B u16) beside the histogram. | greedy prefetch order with the overlap correction must beat the naive order on a fixture with known correlated predicates | +| 24 | room 23 + MAJ3 | **Quorum over correlated bands is a MAJ3 with a correction term.** Three predicates that overlap heavily vote as one; the overlap matrix tells the quorum how many independent votes it actually has. `jc::quorum::QuorumLevel` gets a mask-native input. | three identical masks must yield quorum 1, not 3 | +| 25 | rooms 12 + 24 | **Self-reinforcement without a model, with humility.** The style whose survivors moved least wins (room 12) only if its votes were independent (room 24). The φ⁻¹ ceiling from the free-energy doctrine has a mask reading: never let a program win on fewer than `1/φ` independent votes. | a style that wins only via correlated bands must be blocked by the independence floor | +| 26 | everything above | **The whole stack is one sealed object per (classid, version): `NestedBands` + overlap matrix + rank column.** Search, monitoring, rank, basin, proprioception, drift, novelty, quorum, style selection, episodic recall and time travel are reads of it. There is no second object. | any of the above requiring state outside this object is a plan defect | +| 27 | room 26 | **The object is the Think struct's `trajectory` field, read as masks instead of `Vsa16kF32`.** CLAUDE.md's Think struct carries a trajectory; the 2026-07-10 ruling moved the Markov trajectory off VSA onto the temporal stream. The sealed histogram IS a temporal-stream projection (versions = time). Think.trajectory becomes a `&NestedBands`, zero-copy, one owner. VSA keeps its ≤32-item niche (I-VSA-IDENTITIES); nothing else changes. | the four VSA tests (register laziness, bundle size, orthogonality, cleanup codebook) all answer "no" for the trajectory field, confirming it was never a VSA workload | + +## 4. What is NOT free (carried from the consolidation) + +- Independence assumption in popcount selectivity (room 23 is the fix, unmeasured). +- 16 × 8 KB per histogram per version hot; room 1's u8 column is the cold form. +- Rollover bisection reads the distance column (the one non-mask read). +- `PILLAR_6_PSD_THRESHOLD` 0.10 placeholder blocks room 3 in production. +- `columnar_hop_bench` is the only bench in scope; every row above needs its own. +- No repository code is touched by this plan. It is a proposal with falsifiers. + +## 5. Sequencing (probe-first, per truth-architect) + +1. D-NXG-1 seal shape + D-NXG-2/3 T1 names (edit-only spec, warden pre-spawn). +2. PROBE-NXG-HIST-1: build one `NestedBands` from a real facet column; assert + bucket masks are disjoint, nested, and sum to N; assert rank = partition + point (rooms 0, 16). +3. PROBE-NXG-ROLL-1: rollover on a bimodal column; entropy-triggered vs + budget-triggered split order (room 2, D-NXG-5). +4. PROBE-NXG-FLOOR-1: rank-derived floor vs `mu + 3σ` on the HHTL tiers + fixture; measure false-alarm rate under Jirak-rate tolerance (D-NXG-6). +5. Only after 2–4 are green: rooms 4–8. Rooms 9–27 stay PROPOSAL until then. + +## 6. D-id table + +| D-id | deliverable | status | +|---|---|---| +| D-NXG-1 | `NestedBands` sealed shape (T2), version-keyed, one owner | Queued | +| D-NXG-2 | T1 name audit: `popcount_words` present or minted; bucket = `mask_ternlog::<0x10>` by name | Queued | +| D-NXG-3 | T1 `bisect_column_by_mask` partial-popcount bisection | Queued | +| D-NXG-4 | Prozentrang = bucket index; `shape × rank` computed from the slab | Queued | +| D-NXG-5 | rollover: split on budget or entropy, merge on collapse, never rewrite | Queued | +| D-NXG-6 | rank-derived reject floor; σ demoted to diagnostic | Queued | +| D-NXG-7 | preheat by mask inheritance | Queued | +| D-NXG-8 | one early-exit rule for search and alarm | Queued | +| D-NXG-9 | histogram entropy via `thought_atoms::normalized_entropy` as rollover timer | Queued | +| D-NXG-10 | Mexican hat as two boundaries; Pillar-15 on the histogram | Queued | +| D-NXG-11 | EWA sandwich on histogram-recovered Σ; blocked on Pillar-6 calibration | Blocked | +| D-NXG-12 | overlap matrix + corrected prefetch order + mask-native quorum input | Queued | diff --git a/crates/lance-graph/examples/sigma_probe_masked_traverse.rs b/crates/lance-graph/examples/sigma_probe_masked_traverse.rs new file mode 100644 index 000000000..482792cb5 --- /dev/null +++ b/crates/lance-graph/examples/sigma_probe_masked_traverse.rs @@ -0,0 +1,197 @@ +//! σ-probe for `blasgraph::typed_graph::masked_traverse` — MEASUREMENT ONLY. +//! +//! `masked_traverse` today computes the FULL `A·A` product, then filters every +//! nonzero against a `Vec` label mask and rebuilds a COO. The open +//! question is whether pushing the mask *inside* the product could remove real +//! work, or whether it would only remove the filter+rebuild overhead. +//! +//! That is decided by selectivity, which is measurable BEFORE any inward-mask +//! implementation exists. This probe changes no library code: it calls `mxm` +//! directly for the unmasked baseline and `masked_traverse` for the masked +//! result, and reports per-call rows. +//! +//! Two selectivities, deliberately separated: +//! +//! σ_result = nnz(masked(A²)) / nnz(A²) — result-space compression +//! σ_columns = |M| / N — mask density (column space) +//! +//! Both tiny ⟹ the case for mask pushdown is clean. +//! σ_columns ≪ σ_result ⟹ topology CONCENTRATES into the selected columns, and +//! a naive 1/σ_columns speedup prediction would exaggerate the win. +//! σ_result ≈ 1 ⟹ pushdown cannot remove result work; stop. +//! +//! Distributions: dense random, uniform sparse, and block-clustered — the last +//! matters most, since uniform random sparsity alone is not a sufficient +//! benchmark for graph workloads. The clustered case is run with BOTH a +//! block-aligned mask and a random mask of identical density, which is what +//! isolates concentration from selectivity. +//! +//! cargo run -p lance-graph --release --example sigma_probe_masked_traverse + +use lance_graph::graph::blasgraph::typed_graph::TypedGraph; +use lance_graph::graph::blasgraph::{BitVec, CooStorage, GrBDesc, GrBMatrix, HdrSemiring}; + +/// SplitMix64 — deterministic, so every run measures the same graphs. +struct Rng(u64); + +impl Rng { + fn next_u64(&mut self) -> u64 { + self.0 = self.0.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = self.0; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) + } + /// Uniform in `[0, 1)`, top 53 bits so every draw is exact in `f64`. + fn unit(&mut self) -> f64 { + (self.next_u64() >> 11) as f64 / (1u64 << 53) as f64 + } +} + +/// Edge list -> adjacency matrix. Each edge carries a deterministic BitVec +/// payload so the semiring has something non-trivial to compose. +fn matrix_from_edges(n: usize, edges: &[(usize, usize)]) -> GrBMatrix { + let mut coo = CooStorage::new(n, n); + for (idx, &(r, c)) in edges.iter().enumerate() { + coo.push(r, c, BitVec::random(0x51ED_0000 + idx as u64)); + } + GrBMatrix::from_coo(&coo) +} + +/// Erdos-Renyi: every ordered pair independently present with probability `d`. +fn dist_random(n: usize, d: f64, seed: u64) -> Vec<(usize, usize)> { + let mut rng = Rng(seed); + let mut e = Vec::new(); + for r in 0..n { + for c in 0..n { + if rng.unit() < d { + e.push((r, c)); + } + } + } + e +} + +/// Block-clustered: `blocks` communities, dense inside, sparse across. This is +/// the graph-like case — the one where tile/mask structure is expected to pay, +/// and the one uniform random sparsity cannot stand in for. +fn dist_clustered( + n: usize, + blocks: usize, + d_in: f64, + d_out: f64, + seed: u64, +) -> Vec<(usize, usize)> { + let mut rng = Rng(seed); + let bsz = n / blocks; + let mut e = Vec::new(); + for r in 0..n { + for c in 0..n { + let same = bsz > 0 && (r / bsz) == (c / bsz); + if rng.unit() < if same { d_in } else { d_out } { + e.push((r, c)); + } + } + } + e +} + +/// `count` node ids drawn uniformly without replacement. +fn mask_random(n: usize, count: usize, seed: u64) -> Vec { + let mut rng = Rng(seed); + let mut ids: Vec = (0..n).collect(); + for i in (1..n).rev() { + let j = (rng.next_u64() % (i as u64 + 1)) as usize; + ids.swap(i, j); + } + ids.truncate(count); + ids.sort_unstable(); + ids +} + +/// `count` CONTIGUOUS node ids — i.e. whole communities under `dist_clustered`. +/// Same density as `mask_random`, different alignment: the difference between +/// the two rows is precisely the concentration effect. +fn mask_blocks(count: usize) -> Vec { + (0..count).collect() +} + +/// One measured row. `nnz_full` comes from `mxm` directly; `nnz_masked` from +/// `masked_traverse`, i.e. the shipped path, unmodified. +#[allow(clippy::too_many_arguments)] +fn probe(label: &str, n: usize, edges: &[(usize, usize)], mask_name: &str, mask_ids: &[usize]) { + let mut g = TypedGraph::new(n); + g.add_relation("R", matrix_from_edges(n, edges)); + g.add_label("M", mask_ids); + + let sr = HdrSemiring::XorBundle; + let a = g.relation("R").expect("relation R"); + let full = a.mxm(a, &sr, &GrBDesc::default()); + let nnz_full = full.nnz(); + + let masked = g.masked_traverse("R", "M", &sr).expect("masked_traverse"); + let nnz_masked = masked.nnz(); + + let sigma_result = if nnz_full == 0 { + f64::NAN + } else { + nnz_masked as f64 / nnz_full as f64 + }; + let sigma_columns = mask_ids.len() as f64 / n as f64; + // >1 means the mask's columns hold MORE than their share of the product: + // topology concentrates into the selected region. + let concentration = sigma_result / sigma_columns; + + println!( + "{label:<26} {mask_name:<14} n={n:<5} edges={:<7} nnz_full={nnz_full:<7} \ + nnz_masked={nnz_masked:<7} sigma_result={sigma_result:>7.4} \ + sigma_columns={sigma_columns:>7.4} concentration={concentration:>6.2}x", + edges.len() + ); +} + +fn main() { + // Kept small deliberately: `mxm` is O(N^3) over 2 KB `BitVec` values, so a + // debug build at N=256 does not finish in useful time. Override with + // `SIGMA_PROBE_N` for a --release run. + let n: usize = std::env::var("SIGMA_PROBE_N") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(96); + println!("sigma-probe :: masked_traverse (measurement only, no library change)\n"); + println!("sigma_result = nnz(masked(A^2)) / nnz(A^2)"); + println!("sigma_columns = |M| / N"); + println!("concentration = sigma_result / sigma_columns (>1 = mass concentrates in M)\n"); + + let densities = [(0.02_f64, "2%"), (0.10, "10%"), (0.40, "40%")]; + + // A. dense random + let a = dist_random(n, 0.15, 0xA11CE); + for (frac, tag) in densities { + let ids = mask_random(n, (n as f64 * frac) as usize, 0xD1); + probe("A/dense-random d=0.15", n, &a, tag, &ids); + } + println!(); + + // B. uniform sparse + let b = dist_random(n, 0.02, 0xB0B); + for (frac, tag) in densities { + let ids = mask_random(n, (n as f64 * frac) as usize, 0xD2); + probe("B/uniform-sparse d=0.02", n, &b, tag, &ids); + } + println!(); + + // C. clustered — 16 communities of 16. Random mask first... + let c = dist_clustered(n, 8, 0.50, 0.002, 0xC0FFEE); + for (frac, tag) in densities { + let ids = mask_random(n, (n as f64 * frac) as usize, 0xD3); + probe("C/clustered rand-mask", n, &c, tag, &ids); + } + println!(); + // ...then a BLOCK-ALIGNED mask of identical density. The delta between + // these two groups is the concentration effect, isolated. + for (frac, tag) in densities { + let ids = mask_blocks((n as f64 * frac) as usize); + probe("C/clustered block-mask", n, &c, tag, &ids); + } +} diff --git a/crates/lance-graph/src/graph/blasgraph/clam_neighborhood.rs b/crates/lance-graph/src/graph/blasgraph/clam_neighborhood.rs index e2b334391..6b2d92c24 100644 --- a/crates/lance-graph/src/graph/blasgraph/clam_neighborhood.rs +++ b/crates/lance-graph/src/graph/blasgraph/clam_neighborhood.rs @@ -39,12 +39,12 @@ pub fn scent_l1_distance(a: &[u8], b: &[u8]) -> u32 { /// /// Alternative metric: count differing band bits across all edges. /// More aligned with the boolean lattice structure of scent bytes. +/// +/// Routes through [`super::ndarray_bridge::dispatch_hamming`] rather than +/// re-deriving a scalar `count_ones()` fold — same result, one kernel. pub fn scent_hamming_distance(a: &[u8], b: &[u8]) -> u32 { assert_eq!(a.len(), b.len()); - a.iter() - .zip(b.iter()) - .map(|(&x, &y)| (x ^ y).count_ones()) - .sum() + super::ndarray_bridge::dispatch_hamming(a, b) as u32 } /// A cluster radius observation for conjecture testing. diff --git a/crates/lance-graph/src/graph/blasgraph/hdr.rs b/crates/lance-graph/src/graph/blasgraph/hdr.rs index bc1776801..2e342861c 100644 --- a/crates/lance-graph/src/graph/blasgraph/hdr.rs +++ b/crates/lance-graph/src/graph/blasgraph/hdr.rs @@ -112,15 +112,31 @@ pub struct ShiftAlert { /// Hamming distance between two word arrays (XOR + popcount). /// +/// Routes through [`super::ndarray_bridge::dispatch_hamming`], the single +/// SIMD-dispatched Hamming kernel for this module tree (VPOPCNTDQ → +/// AVX-512BW → AVX2 → scalar, via `ndarray::hpc::bitwise` under +/// `ndarray-hpc`). The previous body was a scalar `count_ones()` loop that +/// bypassed that dispatch — a second implementation of a kernel the crate +/// already owned. +/// +/// XOR-then-popcount is invariant under byte order, so viewing the `u64` +/// words as their little-/big-endian byte image changes nothing: the same +/// bits are compared, only their traversal order differs. +/// /// Both slices must have the same length. #[inline] fn words_hamming(a: &[u64], b: &[u64]) -> u32 { debug_assert_eq!(a.len(), b.len()); - let mut dist = 0u32; - for i in 0..a.len() { - dist += (a[i] ^ b[i]).count_ones(); - } - dist + // SAFETY: `u8` has alignment 1 and no invalid bit patterns, so any + // `[u64]` is a valid `[u8]` of 8× the length; the borrow keeps `a`/`b` + // alive for the call and neither view is written through. + let (a_bytes, b_bytes) = unsafe { + ( + core::slice::from_raw_parts(a.as_ptr().cast::(), core::mem::size_of_val(a)), + core::slice::from_raw_parts(b.as_ptr().cast::(), core::mem::size_of_val(b)), + ) + }; + super::ndarray_bridge::dispatch_hamming(a_bytes, b_bytes) as u32 } /// Sampled Hamming distance: compare every `step`-th word and extrapolate.