Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,49 @@
## 2026-09-04 — lgj_hop: the conjunction is one truth-table pass, and the lane is read as a lane (5×)

**Branch `claude/pr-294-ragged-path-validation-170zcy`**, lgj-abi only — no
ABI symbol, no minor bump, no Java change. Depends on the ndarray PR of the
same branch name (`mask_ternlog` / `mask_ternlog_assign` + the contiguous
fast path in `eq_u32_strided_to_mask`).

**What changed.** `lgj_hop`'s selection `selected = class_f ∧ src ∧ struct_f`
was spelled as two `mask_and_assign` passes through a scratch write. AND is
the rank-1 spelling of a 3-input mask op; the conjunction is now ONE
`mask_ternlog_assign::<AND3>` pass (one `VPTERNLOGQ` per 512 bits) via a new
`kernels::simd_mask_ternlog_assign` wrapper + `kernels::ternlog` re-export
(abi.md §8: exports names `kernels::ternlog::AND3`, never `ndarray::simd`
directly). The `lgj_hop` doc block, which still described the PR #39
gather that R1 (`0385269`) reversed, is rewritten to the shipped mask-
algebra shape with the arc named.

**The number, and where it actually came from.** `columnar_hop_bench`,
65 536 rows, 32 facets, equivalence asserted before timing:

| arm | before | after |
|---|---|---|
| classid (3 933 src) | 6 342 µs | **1 203 µs** |
| hop2 (6 943 src) | 6 408 µs | **1 101 µs** |
| all (65 536 src) | 7 547 µs | **1 851 µs** |

The ternlog wire is one of three mask passes and cannot account for 5×.
The bulk is the ndarray-side finding it exposed: `eq_u32_strided_to_mask`
at `stride_bytes == 4` — every lane of the minor-10 facet-major store —
gathered 16 bounds-checked scalar reads into a temporary before the
`U32x16` compare. The store was contiguous; the kernel still read it as
strided. 16 MB of lane per hop at 2.1 GB/s, now ~13 GB/s. Fixed in the
primitive, so `lgj_op_eq_classid` and every other stride-4 caller gets it
with no change here.

**Gates.** lgj-abi `cargo fmt --check` clean, `cargo test` 138/138 + 3/3
(incl. the pinned 10 → 19 → 29 hop regression on both layouts, the
aliasing and reserved-mode falsifiers), `cargo clippy --all-targets -D
warnings` clean. Java suite not re-run: no Java source or ABI surface
changed; the `.so` must be rebuilt before the next Java run.

**Not in this PR (named, own PR):** the BBB fence — no byte offset,
stride, slot index, or carving width in any public Java signature
(`WideFieldMask.ofFacets(int…)`, the 97 `LgjLaneDesc` lanes, the `*At`
accessors are the leaks). The wall passes handles, classids, names,
counts, statuses.
## 2026-09-04 — the T2/T3 membrane gets its gate: ApiSurfaceTest fences the raw register + names every breach

**Branch `claude/membrane-tiers-bbb-fence`**, test-only. The paired doctrine +
Expand Down
82 changes: 40 additions & 42 deletions native/lgj-abi/src/exports.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1673,39 +1673,30 @@ fn resolve_rowstore_and_hop_masks(
/// `n_rows` as a `u64` BEFORE any `as usize` cast, so an out-of-range
/// `u64` target can never reach an indexing operation.
///
/// **Shape: gather, not sweep.** The hop reads ONLY the rows `src`
/// names. For each such row it walks its participating facets in place
/// out of that row's own 512 bytes — classid compare, payload decode and
/// scatter together, one row's cache lines at a time.
///
/// It did not always. Two earlier shapes swept the WHOLE population and
/// then intersected with `src`: first one full-width
/// `simd_rowstore_classid_mask` pass PER FACET (32 passes), then one
/// `simd_rowstore_facet_match` pass answering all 32 at once. Both were
/// replaced on measurement, not taste
/// (`ISS-LGJ-HOP-SWEEPS-FULL-POPULATION`).
///
/// The crossover a sweep would need in order to win does not exist.
/// `examples/hop_gather_vs_sweep.rs` measured both shapes over four
/// populations (1 024 … 262 144) × twelve frontier densities, asserting
/// byte-identical output at every point: gather wins EVERY configuration,
/// by 2 612× at the sparsest and still **1.7×** at 100 % density. The
/// mechanism is why it holds even when every row is in the frontier — a
/// sweep MATERIALISES an `n`-element per-row intermediate that each row
/// reads exactly once, so the cost is never amortised, while the gather
/// computes the same answer inline. A sweep is strictly more work at
/// every density, not merely more work at sparse ones.
///
/// Consequence worth stating plainly: this scalar gather beats a
/// vectorised `ndarray::simd` sweep. The win is in NOT DOING THE WORK,
/// not in the vector width — so no SIMD primitive is called here, and
/// none is missing. (`simd_rowstore_facet_match` remains the kernel
/// behind [`lgj_row_facet_match`]; it is not orphaned.)
///
/// The one shape that could still favour a precomputed mask is REUSE —
/// memoising the per-row answer across many hops on the same
/// `(store, classid)`. That is a caching design with its own
/// invalidation questions, and is deliberately not this function's.
/// **Shape: selection is mask algebra.** For each participating facet the
/// hop computes two whole-population predicates through the layout's own
/// lane geometry — `class_f` (classid == `edge_classid`, facet base +0) and
/// `struct_f` (`payload_hi32 == 0`, facet base +12), both the same strided
/// equality primitive — and conjoins them with `src` in ONE truth-table
/// pass (`mask_ternlog_assign::<AND3>`). No row is examined to decide
/// whether it participates. The only walk left is the scatter, which
/// EMITS from the selected set: the destination index is decoded from the
/// selected row's payload, the operand of a permutation rather than a
/// membership decision.
///
/// It did not always. A gather (walk `src`'s set bits, read each row's
/// facets in place) measured faster on the AoS store and shipped briefly;
/// the operator ruled it out — walking a population's set bits is a
/// serialization of a population that is already there, whether or not it
/// allocates — and R1 restored the mask shape at a measured 19× cost on
/// AoS. The facet-major columnar store (ABI minor 10) is what makes the
/// lawful shape fast: every predicate becomes a contiguous pass, and the
/// hop runs 3.3–4.8× over AoS at every frontier arm. The two-AND spelling
/// of the conjunction was the last rank-1 residue of that arc.
///
/// The next rung is a semiring product (`dst = src ⊗ A`) over an adjacency
/// operand this ABI does not yet carry — the V3 `EdgeBlock` in the row KEY
/// (decode modes 1..=3, RESERVED).
///
/// `docs/abi.md` §13 is the full normative statement.
#[no_mangle]
Expand Down Expand Up @@ -1762,15 +1753,17 @@ pub extern "C" fn lgj_hop(

// SELECTION IS MASK ALGEBRA.
//
// selected_f = src ∧ class_f struct_f
// selected_f = ternlog<AND3>(class_f, src, struct_f)
// dst = ⋁_{f ∈ participation} scatter(selected_f)
//
// Both predicates are the SAME strided-equality primitive
// (`simd_rowstore_u32_eq_mask`) at two offsets into the facet — the
// classid at +0, the structured-edge gate at +12 — and the two ANDs
// are word-parallel over 64 rows at a time. No row is examined to
// decide whether it participates; participation is computed for the
// whole population and intersected.
// classid at +0, the structured-edge gate at +12 — and their
// conjunction with `src` is ONE 3-input truth-table pass
// (`mask_ternlog_assign::<AND3>`, one VPTERNLOGQ per 512 bits), not
// two ANDs through a scratch write. No row is examined to decide
// whether it participates; participation is computed for the whole
// population and intersected.
//
// Three shapes preceded this one and each traded the algebra for
// arithmetic:
Expand Down Expand Up @@ -1814,12 +1807,17 @@ pub extern "C" fn lgj_hop(
edge_classid,
&mut selected,
);
// ∧ src — narrow to the frontier.
kernels::simd_mask_and_assign(&mut selected, &src_snapshot);
// struct_f — payload_hi32 == 0 marks a structured edge.
kernels::simd_rowstore_u32_eq_mask(bytes, h_off, h_stride, n, 0, &mut structured);
// ∧ — the gate that used to be an `if`.
kernels::simd_mask_and_assign(&mut selected, &structured);
// ∧ src ∧ struct_f — ONE truth-table pass. AND is the rank-1
// spelling of a mask op: `selected & src & struct_f` is the
// 3-input table AND3 (0x80), and spelling it as two `mask_and`
// passes wrote every word twice to say it once.
kernels::simd_mask_ternlog_assign::<{ kernels::ternlog::AND3 }>(
&mut selected,
&src_snapshot,
&structured,
);

// Emit from the SELECTED set. Every row reached here has already
// satisfied all three predicates; the walk decides nothing.
Expand Down
18 changes: 18 additions & 0 deletions native/lgj-abi/src/kernels.rs
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,24 @@ pub fn simd_mask_andnot_assign(dst: &mut [u64], src: &[u64]) {
ndarray::simd::mask_andnot_assign(dst, src);
}

/// The truth-table immediates for [`simd_mask_ternlog_assign`] — re-exported
/// so a call site in `exports` names `kernels::ternlog::AND3`, never reaching
/// past this module for its SIMD vocabulary (abi.md §8).
pub use ndarray::simd::ternlog;

/// `a = ternlog::<IMM>(a, b, c)` — one 3-input Boolean pass over three masks.
///
/// The general member of the mask-op family: `AND3` here is what two
/// consecutive [`simd_mask_and_assign`] calls spell as two passes and one
/// scratch write. On AVX-512 it is one `VPTERNLOGQ` per 512 bits; the
/// polyfill elsewhere. Tail contract per `ndarray::simd::mask_ternlog`: for
/// the even (all-zero → 0) tables, which every named immediate is, a
/// conforming input tail stays zero.
#[inline]
pub fn simd_mask_ternlog_assign<const IMM: i32>(a: &mut [u64], b: &[u64], c: &[u64]) {
ndarray::simd::mask_ternlog_assign::<IMM>(a, b, c);
}

/// Sum of `values[i]` over set mask bits, widened to `i64`.
#[inline]
pub fn simd_masked_sum_i32(values: &[i32], mask_words: &[u64]) -> i64 {
Expand Down