diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 2023ea5bd..d7811bf79 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,64 @@ +## 2026-09-02 — E-QUALIA-IS-RANK-INERT-AT-THE-FRONTIER-AND-POPULATION-LOSES-TO-COUNTING-1 + +**Status:** MEASURED (PROBE-POP-READOUT-1 / D-POP-1, `pop_readout.rs`, whole +KJV, 227,261 candidates, 9 splits x 25 shuffles, deterministic). +**Confidence:** High on findings 1/3/4 (each is a direct measurement with a +size-preserving null or an exact identity); Medium on 2 (a weak effect, real +against its null, on one corpus and one label). + +**The pre-registered claim was KILLED.** A population-typicality readout (an +object's Cam96 distance to its subject's own basin centroid) does not improve +the exploration frontier's ranking: mean precision@10 falls 0.289 -> 0.011 +against the shipped `FrontierEdge::curiosity`, and lands BELOW its own +label-shuffle null (null p95 0.133). Combining it in is worse than omitting it. + +**It is not noise, though.** Controlling for prefix frequency, the readout's +partial Spearman against later recurrence is 0.090 against a null 95th +percentile of 0.020. A weak global trend and a useless top-k coexist, because +the extreme of the readout is degenerate (objects sitting on their own +centroid). Precision@k probes the tail, Spearman the trend; when they +disagree, that disagreement is the result. + +**The sharpest finding is about shipped code, not about the hypothesis: +`curiosity_gestalt` cannot reorder a frontier.** Measured +`spearman(curiosity, gestalt_magnitude) = 1.000000` under two deliberately +contrasting `MulAssessment`s. `exploration.rs:180-215` shows why -- the +magnitude is `base * fw * dk * flow * trust * staunen_boost * ground_gate` +and every factor but `base` is per-GRAPH, so all per-edge variation flows +through `curiosity` alone. MUL and qualia rescale the frontier; they never +reorder it. Consequently the operator's framing question, "does population +beat qualia/context alone", had an a-priori answer for any RANKING task: +qualia carries zero ranking information at the frontier as shipped. Whether +that is a defect or an intended global gain is a design question this entry +does not settle -- but a weighting that cannot change an order cannot be +evidence for or against a competing readout either. + +**And the control beat everything cognitive.** Plain prefix frequency reaches +precision@10 = 0.756 against curiosity's 0.289, while curiosity is +ANTI-correlated with recurrence (rho ~ -0.27) -- consistent with it working as +designed (prefer the rare and unqueried) rather than failing. The rule this +banks: any future frontier-ranking claim clears the frequency control first, +or it has measured nothing. This is the same lesson +`E-BASIN-WIDTH-IS-N-ARTIFACT-1` banked for held-out reliability, applied to +ranking: a quantity that co-varies with sample count needs a partial, not a +raw correlation. The frequency arm and the partial were a deliberate +STRENGTHENING of the pre-registered design, added before the run and recorded +as such in the plan. + +**A survey error, corrected by trying to run it:** the plan named Fisher-z and +`RollingFloor::occupancy` as available ingredients. They are not reachable -- +`helix` is not a dependency of `deepnsm-v2`, and adding one pulls the ndarray +git fork into that crate's build. The inventory listed primitives without +checking the dependency edge. Under rank combination Fisher-z is inert anyway +(a monotone transform cannot change a rank), so its disable arm is answered +for a rank readout and open only for a magnitude one. + +**Consequence for the six families:** family 3 as a molecule feeding frontier +selection is NOT licensed; a carrier is licensed even less. The vacancy +stands. The readout's plausible home is a basin-level prior on the global +trend rather than a top-k selector -- stated as a question, per the +falsifier-first rule, not as a direction. + ## 2026-09-02 — E-SIX-SEMANTIC-FAMILIES-MUST-NOT-IMPERSONATE-EACH-OTHER-1 **Status:** OPERATOR RULING (semantic-family recovery), landed as the diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index b7dce620b..015f9ecf7 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -1,3 +1,9 @@ +## 2026-09-02 — branch (D-POP-1 result): PROBE-POP-READOUT-1 — INVENTORY DELTA + +- ADDED `crates/deepnsm-v2/examples/pop_readout.rs` — the D-POP-1 probe (no library surface, no new type, no tenant). Consumes `Cam96Space::distance`, `basin_self_code`, `partial_spearman` and `lance_graph_contract::exploration::{FrontierEdge, NarsTruth}`; produces a KILL verdict plus three side-findings (plan §6a). +- MEASURED, not added: `FrontierEdge::curiosity_gestalt` is RANK-INERT — its magnitude is a per-graph scalar multiple of `curiosity` (`exploration.rs:180-215`), so MUL/qualia cannot reorder a frontier. Any consumer treating the gestalt magnitude as a distinct ranking key is using a rescale. +- UNCHANGED: no contract type added, removed or re-read. Family 3 remains a vacancy. + ## 2026-09-02 — branch (cleanup, no PR yet): semantic-family recovery — CONTRACT INVENTORY DELTA - ⊘ REMOVED `lance_graph_contract::epistemic_bassin` (all of it: `EpistemicBassin24`, `AxisState`, the Belnap masks, `sweep_ternlog`/`eval_ternlog`, `info_gain_u4`, `sigma_tension_u4`, `loco_band`, `axes`) and `lance_graph_contract::basin_lanes` — operator semantic-family ruling 2026-09-02 (`E-SIX-SEMANTIC-FAMILIES-MUST-NOT-IMPERSONATE-EACH-OTHER-1`). The 2026-09-01 inventory rows below are regraded ⊘ in place; nothing they named reached a tenant, a layout version or a producer. diff --git a/.claude/board/PR_ARC_INVENTORY.md b/.claude/board/PR_ARC_INVENTORY.md index 0f7ce7f62..79e7218e3 100644 --- a/.claude/board/PR_ARC_INVENTORY.md +++ b/.claude/board/PR_ARC_INVENTORY.md @@ -10,6 +10,21 @@ > census §8.3 trap 10: read the body FIRST, then open for write — never > inline both in one expression. +## 2026-09-02 — lance-graph branch `claude/medcare-rs-continue-6nhbxn` (D-POP-1 result PR, after #1135) — PROBE-POP-READOUT-1 + +- **Added:** `crates/deepnsm-v2/examples/pop_readout.rs` (probe only, no library + surface); plan §6a RESULT; EPIPHANIES entry; LATEST_STATE delta; STATUS_BOARD + D-POP-1 -> Shipped (KILL). +- **Measured:** VERDICT KILL on the pre-registered claim; `curiosity_gestalt` + rank-inertness at rho = 1.000000 over 227,261 candidates; frequency control + p@10 = 0.756 vs the shipped ranker's 0.289; population partial rho 0.090 vs + null p95 0.020. +- **Deferred:** the Fisher-z and RollingFloor legs (helix unreachable from + deepnsm-v2); any population carrier, tenant, ClassView or axis set -- the + vacancy stands and this result narrows rather than widens it. +- **Confidence:** High for the KILL and the two identity/control findings; + Medium for the weak non-null trend (one corpus, one label). + ## 2026-09-02 — lance-graph branch `claude/medcare-rs-continue-6nhbxn` (survey PR, opened after #1134) — post-teardown buildup survey v1 - **Added:** `.claude/plans/post-teardown-buildup-survey-v1.md` (survey, diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 049afa5d4..adaf69dbe 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -3,7 +3,7 @@ | D-id | deliverable | status | |---|---|---| | D-POP-0 | the survey itself: six-family inventory, atom/operator inventory, family×operator legality matrix, molecule candidates, gap list (plan §§1–5) | **Shipped (plan-only, this PR)** | -| D-POP-1 | PROBE-POP-READOUT-1: composed population readout (Cam96 centroid distance → Fisher-z → occupancy bucket) vs `curiosity` / `curiosity_gestalt` at predicting later-confirmed frontier edges on the KJV stream; size-preserving shuffle null; PASS/KILL pre-registered in plan §6 | **Queued** — needs the `v0.1.0-cam96-data` + Gutenberg #10 fetch; one example binary, no new types | +| D-POP-1 | PROBE-POP-READOUT-1: composed population readout (Cam96 centroid distance → Fisher-z → occupancy bucket) vs `curiosity` / `curiosity_gestalt` at predicting later-confirmed frontier edges on the KJV stream; size-preserving shuffle null; PASS/KILL pre-registered in plan §6 | **Shipped — VERDICT KILL** (`pop_readout.rs`, 2026-09-02; `E-QUALIA-IS-RANK-INERT-AT-THE-FRONTIER-AND-POPULATION-LOSES-TO-COUNTING-1`, plan §6a). Population readout degrades frontier p@10 0.289 -> 0.011, below its own shuffle null; real partial rho 0.090 vs null p95 0.020 (weak but non-null global trend). Side-findings: `curiosity_gestalt` is rank-inert (rho = 1.000000 vs `curiosity` over 227,261 candidates, two assessments); plain frequency p@10 = 0.756 beats every cognitive arm; helix (Fisher-z / RollingFloor) is not reachable from deepnsm-v2, so those legs are NOT RUN | | D-POP-2 | contradiction write-back producer (`elect_peers` → `WitnessLens::write_register`) — the one cheap gap that unblocks the contradiction-driven-revision molecule (plan §4) | **Queued** — not before D-POP-1 reports | ## dismech-causal-replay-v1 diff --git a/.claude/board/SUPERSESSION-INDEX.md b/.claude/board/SUPERSESSION-INDEX.md index 74c236914..9d51051aa 100644 --- a/.claude/board/SUPERSESSION-INDEX.md +++ b/.claude/board/SUPERSESSION-INDEX.md @@ -92,7 +92,7 @@ a licence to act on it. | **READ** | `mul-consumer-census-v1` | `GateDecision` | MEASUREMENT COMPLETE — 2026-08-27. Measureme | 1/2 | | **READ** | `mul-ewa-trust-propagation-v1` | `GateDecision` | PROPOSED — PLAN/BOARD ONLY. Measure-before-c | 1/3 | | **READ** | `persistence-artifact-backed-commit-v1` | `persist_cycle` | RATIFIED (operator ruling 2026-08-09). Phase | 0/0 | -| **READ** | `post-teardown-buildup-survey-v1` | `StepMask` | SURVEY, read-only, plan-only (no code, no te | 3/4 | +| **READ** | `post-teardown-buildup-survey-v1` | `StepMask` | SURVEY, read-only, plan-only (no code, no te | 4/4 | | **READ** | `r2il-bpe-typed-genetic-recombination-v1` | `GateDecision` | PROPOSAL, §7's three falsifiers now RUN (see | 1/1 | | **READ** | `scientific-kg-substrate-v1` | `GateDecision` | PROPOSED — **scoping doc**, no code. Records | 8/9 | | **READ** | `self-reasoning-substrate-v1` | `GateDecision` | PROPOSED — doc-only. No code, no contract ch | 5/5 | diff --git a/.claude/knowledge/never-truncate-a-file-you-still-need-to-read.md b/.claude/knowledge/never-truncate-a-file-you-still-need-to-read.md index ff0bba28a..9238a1082 100644 --- a/.claude/knowledge/never-truncate-a-file-you-still-need-to-read.md +++ b/.claude/knowledge/never-truncate-a-file-you-still-need-to-read.md @@ -57,6 +57,34 @@ correctness or ordering — a same-length corruption passes this check. A non-blocking PreToolUse guard (`.claude/hooks/anti-pattern-matching.sh`) injects this rule when a Bash command matches the write-while-reading shape. +## Recurrence — 2026-09-02, EPIPHANIES.md, 25,172 -> 61 lines + +The rule fired again, in a session that had this file's own prohibition in its +context. A board-hygiene pass prepended a new entry with +`open(p,'w').write(E + open(p).read())` -- the exact prohibited shape -- +inside a Python heredoc that ALSO carried two CORRECT prepends +(`b = open(p).read()` first, then `open(p,'w')`) for LATEST_STATE and +PR_ARC_INVENTORY. Writing the safe form twice in the same script did not +prevent writing the unsafe form once. + +What caught it: the mandatory post-write `wc -l` comparison against +`origin/main`, printed for every touched board file in the same command. The +line `EPIPHANIES.md main=25172 now=61` was unmissable, and restoration was a +`git checkout` plus a re-prepend, because the destruction happened in the +working tree and was never committed. + +Two lessons this recurrence adds: + +1. **The guard has to be in the same breath as the write.** The rule as + stated is a prohibition an author must remember; the `wc -l` check is a + detector that runs whether or not they remembered. Print the before/after + line counts in the SAME command that writes, every time, and never commit a + board pass without reading that output. +2. **Mixed-safety scripts are the dangerous shape.** A heredoc containing + several prepends is where this hides: the correct ones make the script + look reviewed. Prefer one helper used for every prepend in a pass over + three hand-written ones. + ## Cross-reference / retrieval footer - Restore PR: lance-graph **#1082** (merge `82679c3a`); prohibition PR **#1083** diff --git a/.claude/plans/post-teardown-buildup-survey-v1.md b/.claude/plans/post-teardown-buildup-survey-v1.md index 79e4161af..4a5ab6344 100644 --- a/.claude/plans/post-teardown-buildup-survey-v1.md +++ b/.claude/plans/post-teardown-buildup-survey-v1.md @@ -241,3 +241,97 @@ name as storage. held-out figures are 0.766 / 0.624. - The tracers reported the working-tree HEADs as `f3eb2f6` (lance-graph) and `1eb2ddb` (OGAR); both are the trees the merges `20eaf7f` / `954fc52` carry. + +--- + +## 6a. PROBE-POP-READOUT-1 — RESULT: **KILL** (measured 2026-09-02) + +Shipped as `crates/deepnsm-v2/examples/pop_readout.rs`. Deterministic; 89 s on +the whole book. Inputs: `bible_wave --export` (70,393 triples over 31,102 +verses) plus the trained `v0.1.0-cam96-data` codebook (12,543 words, 12 axes). +9 split points × 25 shuffles; 227,261 candidates pooled. + +``` +cargo run --manifest-path crates/deepnsm-v2/Cargo.toml --example bible_wave -- pg10.txt --export spo.tsv +cargo run --manifest-path crates/deepnsm-v2/Cargo.toml --example pop_readout -- spo.tsv +``` + +| arm | mean p@10 | mean p@25 | mean p@100 | ρ vs label | +|---|---|---|---|---| +| A0 `curiosity` (shipped ranker) | 0.289 | 0.173 | 0.249 | ≈ −0.27 | +| A1 `curiosity_gestalt` (assessment A) | 0.289 | 0.173 | 0.249 | ≈ −0.27 | +| A1B `curiosity_gestalt` (assessment B) | 0.289 | 0.173 | 0.249 | ≈ −0.27 | +| **AF frequency (control)** | **0.756** | **0.751** | **0.674** | **+0.27** | +| AP population readout alone | 0.011 | 0.036 | 0.076 | +0.09 | +| A2 = A0 + AP (rank mean) | 0.011 | 0.022 | 0.069 | −0.06 | +| A3 = A1 + AP (rank mean) | 0.011 | 0.022 | 0.069 | −0.06 | + +Decisive statistics: + +| quantity | value | +|---|---| +| real partial ρ(AP, label \| freq) | **0.090** | +| null partial ρ — mean / 95th pct | −0.018 / 0.020 | +| mean(A2 p@10) − mean(A0 p@10) | **−0.278** | +| mean(A2 p@10) vs its null mean / 95th pct | 0.011 vs 0.031 / 0.133 | +| mean(AP p@10) vs its null mean / 95th pct | 0.011 vs 0.034 / 0.144 | + +Pre-registered rule: PASS iff (a) real partial ρ > null p95 + 0.02 **AND** +(b) Δp@10 ≥ 0.05 and A2 p@10 above its null p95. **(a) passes, (b) fails +decisively → VERDICT KILL.** + +### The four findings + +1. **KILL on the pre-registered claim.** The population readout does not + improve frontier ranking; it degrades it, 0.289 → 0.011 at p@10, and lands + *below its own shuffle null* (0.011 vs a null p95 of 0.133). Combining it + into the ranking is worse than not having it. +2. **The signal is nevertheless real and null-surviving.** Controlling for + frequency, "typical for its subject" carries partial ρ = 0.090 against a + null p95 of 0.020. A weak global monotone trend and a useless top-k coexist: + the extreme of `−pop` is degenerate (objects sitting essentially *on* their + centroid) while the overall ordering still leans the right way. Precision@k + probes the tail; Spearman probes the trend; they disagree here, and the + disagreement is the finding, not an error. +3. **Qualia is rank-inert at the frontier — measured, not argued.** + `spearman(A0, A1) = 1.000000` and `spearman(A0, A1B) = 1.000000` pooled over + 227,261 candidates under two deliberately contrasting `MulAssessment`s. + Reading `exploration.rs:180-215` says why: `magnitude = base · fw · dk · + flow · trust · staunen_boost · ground_gate`, and every factor except `base` + is per-GRAPH, identical for every candidate. `curiosity_gestalt` can + rescale a frontier; it can never reorder one. So "does population beat + qualia/context alone" had an a-priori answer for any ranking task: qualia + contributes exactly zero ranking information at the frontier today. +4. **Plain counting dominates every cognitive arm.** Prefix frequency reaches + p@10 = 0.756 against the shipped ranker's 0.289 — 2.6× — and the shipped + ranker is *anti*-correlated with recurrence (ρ ≈ −0.27). That is consistent + with `curiosity` working as designed (it prefers the rare and unqueried, and + rare things do not recur) rather than being broken; but it means the + frontier ranker is not selecting for what the corpus goes on to confirm, and + any future ranking claim must clear the frequency control first. + +### Honest limits + +- One corpus (KJV), one label (exact-triple recurrence), one basin definition + (a subject's outgoing objects). This KILLs "a population readout improves + frontier ranking on the recurrence label"; it does not show population + geometry is useless, and recurrence is not what `curiosity` is built to + maximise. +- The codebook is Bible-vocabulary and held-out ρ 0.766 + (`E-CAM96-REVIEW-CORRECTIONS-1`; never cite the crate doc's in-sample 0.828). +- The Fisher-z and `RollingFloor`-occupancy legs the plan named were **NOT + RUN**, for a reason the survey missed: `helix` is not a dependency of + `deepnsm-v2`, and adding one pulls the ndarray git fork into this crate's + build. Under a rank-based combination Fisher-z is analytically inert anyway + (a strictly monotone transform cannot change a rank), so the plan's + Fisher-z disable arm is answered for a rank readout and remains open only + for a magnitude-valued one. + +### Consequence + +Family 3 as a **molecule feeding frontier selection** is NOT licensed by this +measurement — and a carrier is licensed even less than before. The vacancy +stands. What the result does license is a narrower next question, stated as a +question and not a direction: the readout's honest home may be the global +trend (a basin-level prior) rather than a top-k selector, and any such probe +must carry the frequency control and the shuffle null from the start. diff --git a/crates/deepnsm-v2/examples/pop_readout.rs b/crates/deepnsm-v2/examples/pop_readout.rs new file mode 100644 index 000000000..13d750e3b --- /dev/null +++ b/crates/deepnsm-v2/examples/pop_readout.rs @@ -0,0 +1,751 @@ +//! `pop_readout` — does population-typicality (an object's distance to its +//! subject's own basin centroid) improve the exploration-frontier RANKING +//! function beyond what `FrontierEdge::curiosity` already gives? +//! +//! PROBE-POP-READOUT-1, plan `.claude/plans/post-teardown-buildup-survey-v1.md` +//! §6, D-POP-1. +//! +//! ## What is actually under test +//! +//! This probe evaluates the RANKING FUNCTION +//! [`lance_graph_contract::exploration::FrontierEdge::curiosity`] (and its MUL- +//! weighted sibling [`FrontierEdge::curiosity_gestalt`]) — which is exactly what +//! [`MassExplorer::next_frontier_edge`] (upstream, `lance-graph-planner`) sorts +//! its frontier by — rather than the whole `MassExplorer` fetch/extract/revise +//! loop. TD-EXPLORATION-1 leaves `MassExplorer::from_graph`'s frontier +//! structurally empty on this repo's real corpora, so the loop cannot be run +//! end-to-end here; the RANKING is the one piece both reachable and directly +//! testable in isolation, and it is the piece a population readout could +//! actually improve (it changes which edge gets picked next, not what the +//! fetch/extract machinery does with it). +//! +//! ## The label +//! +//! A candidate `(s, p, o)` triple mined from the PREFIX of a book (verses +//! before a split point) is labelled `1.0` iff the exact same `(s, p, o)` +//! recurs in the SUFFIX (verses at or after the split) — i.e. does ranking +//! this candidate highly help find something the rest of the book actually +//! confirms. +//! +//! ## Strengthening beyond the pre-registered design: the frequency control +//! +//! The plan's original arms are `curiosity` (A0) and `curiosity_gestalt` +//! magnitude (A1) vs. the population readout (AP). Both `curiosity` and `pop` +//! are functions of quantities that co-vary with SAMPLE COUNT `n` (how often a +//! candidate was actually observed in the prefix): `curiosity`'s `novelty` term +//! is `1/(n+1)`-shaped, and `n` also drives the confidence that seeds a +//! candidate's basin membership. A raw cross-candidate correlation between any +//! such quantity and a recurrence label is therefore vulnerable to exactly the +//! member-count ARTIFACT `E-BASIN-WIDTH-IS-N-ARTIFACT-1` (and its restatement in +//! `bible_wave.rs`'s own G-SRS3b-3 leg, "does the composite predict beyond +//! size?") measured for basin width: a quantity that merely tracks `n` will +//! look predictive of anything that itself correlates with `n`, with no +//! semantic content behind it. This probe therefore adds, beyond the +//! pre-registered design: an explicit frequency arm `AF = n`, and the decisive +//! statistic is [`partial_spearman`] of the population readout against the +//! label, controlling for frequency — the same instrument `bible_wave.rs` +//! already uses for its own size-confound (G-SRS3b-3). This is a +//! strengthening of the pre-registered design, not a deviation from it: every +//! pre-registered arm and metric is still computed and reported unchanged. +//! +//! ## What this probe does NOT run, and why +//! +//! The plan additionally names a Fisher-z leg and a RollingFloor-occupancy leg +//! (`helix`). Neither runs here, for two separate reasons: +//! +//! 1. `helix` is not a dependency of `deepnsm-v2`. Adding one here would pull +//! the `ndarray` git fork into this crate's build graph for a probe whose +//! whole point is to stay inside the already-available contract + this +//! crate's own dependency set. +//! 2. Under a RANK-based combination (which is what every metric in this probe +//! is — precision@k, Spearman, partial Spearman), a Fisher-z transform is a +//! STRICTLY MONOTONE reparameterization of a correlation coefficient. A +//! strictly monotone transform cannot change any RANKING it is folded into, +//! so Fisher-z is analytically inert for a rank-based readout; it would only +//! matter for a MAGNITUDE-valued combination rule, which this probe does not +//! use. +//! +//! ## Run +//! +//! ```sh +//! cargo run --manifest-path crates/deepnsm-v2/Cargo.toml --example pop_readout -- +//! ``` +//! +//! `` is the 7-column export `bible_wave --export ` writes +//! (`subject_id \t subject_word \t predicate_id \t predicate_word \t object_id +//! \t object_word \t verse`); only the id and verse columns are read here. + +use lance_graph_contract::exploration::{FrontierEdge, NarsTruth}; +use lance_graph_contract::mul::{ + DkPosition, FlowState, Homeostasis, MulAssessment, TrustQualia, TrustTexture, +}; +use lance_graph_contract::sensorium::GraphSignals; + +use deepnsm_v2::{basin_self_code, load_cam96_codes, load_cam96_space, partial_spearman, Cam96}; +use std::collections::HashSet; +use std::path::PathBuf; + +/// The trained artifacts are NOT committed — they ship as the +/// `AdaWorldAPI/lance-graph` release `v0.1.0-cam96-data` (see `data/README.md` +/// for the fetch commands). Loaded at runtime from `data/` (override the +/// directory with `DEEPNSM_V2_DATA`). Copied verbatim from `bible_wave.rs` +/// (examples cannot import each other's helpers). +fn data_file(name: &str) -> Vec { + let dir = std::env::var("DEEPNSM_V2_DATA") + .map(PathBuf::from) + .unwrap_or_else(|_| PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("data")); + let path = dir.join(name); + std::fs::read(&path).unwrap_or_else(|e| { + panic!( + "missing {} ({e}) — fetch the v0.1.0-cam96-data release assets per data/README.md", + path.display() + ) + }) +} + +const MIN_BASIN: usize = 6; +const SPLITS: &[f64] = &[0.30, 0.35, 0.40, 0.45, 0.50, 0.55, 0.60, 0.65, 0.70]; +const SHUFFLES: usize = 25; + +/// One row of the input export: (verse, subject_id, predicate_id, object_id). +type Row = (u32, u16, u16, u16); + +/// One candidate triple with its measured prefix support count. +#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] +struct Candidate { + s: u16, + p: u16, + o: u16, +} + +fn main() { + let path = std::env::args() + .nth(1) + .expect("usage: pop_readout (from `bible_wave --export `)"); + let raw = std::fs::read_to_string(&path).expect("read spo.tsv"); + + // ── load rows ──────────────────────────────────────────────────────── + let mut rows: Vec = Vec::new(); + for (lineno, line) in raw.lines().enumerate() { + if line.is_empty() { + continue; + } + let f: Vec<&str> = line.split('\t').collect(); + assert!( + f.len() == 7, + "line {}: expected 7 tab-separated columns, got {} — is this a \ + `bible_wave --export` TSV?", + lineno + 1, + f.len() + ); + let s: u16 = f[0] + .parse() + .unwrap_or_else(|e| panic!("line {}: bad subject_id {:?}: {e}", lineno + 1, f[0])); + let p: u16 = f[2] + .parse() + .unwrap_or_else(|e| panic!("line {}: bad predicate_id {:?}: {e}", lineno + 1, f[2])); + let o: u16 = f[4] + .parse() + .unwrap_or_else(|e| panic!("line {}: bad object_id {:?}: {e}", lineno + 1, f[4])); + let v: u32 = f[6] + .parse() + .unwrap_or_else(|e| panic!("line {}: bad verse {:?}: {e}", lineno + 1, f[6])); + rows.push((v, s, p, o)); + } + assert!(!rows.is_empty(), "input TSV has no rows"); + let max_verse = rows.iter().map(|&(v, ..)| v).max().unwrap(); + println!( + "LOAD {} rows, max_verse={} ({})", + rows.len(), + max_verse, + path + ); + + // ── load vocab + trained codebook (routing not needed; codes are) ───── + let vocab_text = String::from_utf8(data_file("bible_vocab.txt")).expect("utf8 vocab"); + let mut vocab = deepnsm_v2::PaletteVocab::new(); + vocab.from_frequency_ranked(vocab_text.lines()); + let space = load_cam96_space(&data_file("cam96_codebook.bin")).expect("codebook artifact"); + let codes = load_cam96_codes(&data_file("cam96_codes.bin")).expect("codes artifact"); + assert_eq!( + codes.len(), + vocab.len(), + "codes/vocab misaligned: {} codes, {} vocab words", + codes.len(), + vocab.len() + ); + println!("LOAD trained codebook: {} words, 12 axes", vocab.len()); + + // ── the two MUL assessments A1 (baseline) vs A1B (contrasting) ──────── + // A1: a calibrated, in-flow, moderately-autonomous assessment. + let assess_a = MulAssessment { + trust: TrustQualia { + value: 0.75, + texture: TrustTexture::Calibrated, + }, + dk_position: DkPosition::SlopeOfEnlightenment, + homeostasis: Homeostasis { + flow_state: FlowState::Flow, + allostatic_load: 0.3, + }, + complexity_mapped: true, + free_will_modifier: 0.7, + }; + // A1B: a starkly different reading — overconfident, anxious, low free will — + // to check whether curiosity_gestalt's magnitude can actually re-order a + // frontier under a genuinely different MUL state, not just rescale it. + let assess_b = MulAssessment { + trust: TrustQualia { + value: 0.20, + texture: TrustTexture::Overconfident, + }, + dk_position: DkPosition::MountStupid, + homeostasis: Homeostasis { + flow_state: FlowState::Anxiety, + allostatic_load: 0.9, + }, + complexity_mapped: false, + free_will_modifier: 0.1, + }; + let signals = GraphSignals::default(); + + // ── per-split results, aggregated at the end ─────────────────────────── + struct SplitResult { + arms: Vec, + partial_real: f32, + null_partial_mean: f32, + null_partial_p95: f32, + null_p10_ap_mean: f32, + null_p10_ap_p95: f32, + null_p10_a2_mean: f32, + null_p10_a2_p95: f32, + } + struct ArmResult { + name: &'static str, + p10: f32, + p25: f32, + p100: f32, + spearman: f32, + } + + let mut all_splits: Vec = Vec::new(); + + for &frac in SPLITS { + let cut = (max_verse as f64 * frac) as u32; + + let prefix: Vec<&Row> = rows.iter().filter(|&&(v, ..)| v < cut).collect(); + let suffix_set: HashSet<(u16, u16, u16)> = rows + .iter() + .filter(|&&(v, ..)| v >= cut) + .map(|&(_, s, p, o)| (s, p, o)) + .collect(); + + // objects[s] = distinct object ids seen for subject s in the prefix, + // sorted ascending; edges[s] = the subject's (predicate, object) pairs + // in prefix order (both needed by `basin_self_code`). + use std::collections::HashMap; + let mut objects: HashMap> = HashMap::new(); + let mut edges: HashMap> = HashMap::new(); + // n(s,p,o) = number of prefix rows equal to that exact triple. + let mut counts: HashMap<(u16, u16, u16), u32> = HashMap::new(); + for &&(_, s, p, o) in &prefix { + edges.entry(s).or_default().push((p, o)); + let objs = objects.entry(s).or_default(); + if !objs.contains(&o) { + objs.push(o); + } + *counts.entry((s, p, o)).or_insert(0) += 1; + } + for objs in objects.values_mut() { + objs.sort_unstable(); + } + + // Candidate set: distinct (s,p,o) in prefix whose subject has + // objects[s].len() >= MIN_BASIN. Deterministic ascending (s,p,o) order. + let mut candidates: Vec = counts + .keys() + .filter(|&&(s, _, _)| objects.get(&s).map_or(0, Vec::len) >= MIN_BASIN) + .map(|&(s, p, o)| Candidate { s, p, o }) + .collect(); + candidates.sort_unstable(); + + // Basins: one per eligible subject, computed once and reused per + // candidate of that subject. + let mut basin_of: HashMap> = HashMap::new(); + for &s in objects.keys() { + if objects[&s].len() < MIN_BASIN { + continue; + } + let member_codes: Vec = objects[&s].iter().map(|&o| codes[o as usize]).collect(); + let b = basin_self_code(&space, s, &member_codes, &edges[&s]); + basin_of.insert(s, b); + } + + // Drop candidates whose subject's basin came back None (members empty + // — cannot happen given the MIN_BASIN>=6 filter, but guard anyway). + candidates.retain(|c| basin_of.get(&c.s).is_some_and(Option::is_some)); + + assert!( + !candidates.is_empty(), + "split frac={frac}: zero eligible candidates — MIN_BASIN={MIN_BASIN} too strict \ + for this corpus, or the split point leaves too small a prefix" + ); + + let n_total = candidates.len(); + let labels: Vec = candidates + .iter() + .map(|c| { + if suffix_set.contains(&(c.s, c.p, c.o)) { + 1.0 + } else { + 0.0 + } + }) + .collect(); + let n_pos = labels.iter().filter(|&&l| l > 0.5).count(); + let base_rate = n_pos as f32 / n_total as f32; + assert!( + base_rate > 0.0 && base_rate < 1.0, + "split frac={frac}: base rate {base_rate} is degenerate (all-positive or \ + all-negative labels) — {n_pos}/{n_total}" + ); + + // ── per-candidate: n, pop, and the FrontierEdge-based scores ─────── + let freqs: Vec = candidates + .iter() + .map(|c| *counts.get(&(c.s, c.p, c.o)).unwrap() as f32) + .collect(); + + let pop_of = + |c: &Candidate, basin_lookup: &HashMap>| -> f32 { + let b = basin_lookup[&c.s].as_ref().expect("filtered to Some above"); + space.distance(&codes[c.o as usize], &b.self_code) + }; + let pop: Vec = candidates.iter().map(|c| pop_of(c, &basin_of)).collect(); + let pop_min = pop.iter().cloned().fold(f32::INFINITY, f32::min); + let pop_max = pop.iter().cloned().fold(f32::NEG_INFINITY, f32::max); + assert!( + pop_min < pop_max, + "split frac={frac}: population readout is CONSTANT ({pop_min}) across all \ + {n_total} candidates — the basin/codebook signal is not varying" + ); + + let edges_for: Vec = candidates + .iter() + .map(|c| FrontierEdge { + source: vocab.word(c.s).unwrap_or("?").to_string(), + target: vocab.word(c.o).unwrap_or("?").to_string(), + label: vocab.word(c.p).unwrap_or("?").to_string(), + truth: NarsTruth { + frequency: 1.0, + confidence: *counts.get(&(c.s, c.p, c.o)).unwrap() as f32 + / (*counts.get(&(c.s, c.p, c.o)).unwrap() as f32 + 1.0), + }, + query_count: 0, + is_seed: false, + }) + .collect(); + + let a0: Vec = edges_for.iter().map(FrontierEdge::curiosity).collect(); + let a1: Vec = edges_for + .iter() + .map(|e| e.curiosity_gestalt(&assess_a, &signals).magnitude) + .collect(); + let a1b: Vec = edges_for + .iter() + .map(|e| e.curiosity_gestalt(&assess_b, &signals).magnitude) + .collect(); + let af: Vec = freqs.clone(); + let ap: Vec = pop.iter().map(|&p| -p).collect(); + let a2: Vec = combine_ranks(&a0, &ap); + let a3: Vec = combine_ranks(&a1, &ap); + + let arm = |name: &'static str, scores: &[f32]| -> ArmResult { + ArmResult { + name, + p10: precision_at(scores, &labels, 10), + p25: precision_at(scores, &labels, 25), + p100: precision_at(scores, &labels, 100), + spearman: spearman(scores, &labels), + } + }; + let arms = vec![ + arm("A0 curiosity ", &a0), + arm("A1 gestalt(assessA) ", &a1), + arm("A1B gestalt(assessB) ", &a1b), + arm("AF frequency ", &af), + arm("AP -pop ", &ap), + arm("A2 A0+AP ranks ", &a2), + arm("A3 A1+AP ranks ", &a3), + ]; + + // ── decisive statistic: partial Spearman(AP, label | frequency) ──── + let partial_real = partial_spearman(&ap, &labels, &freqs); + + // ── null: permute which word owns which code (SplitMix64/Fisher- + // Yates), destroying the word↔meaning binding while preserving the + // code multiset exactly. Recompute basins/pop/AP/A2/partial-Spearman + // under the shuffle. ── + let mut null_partials: Vec = Vec::with_capacity(SHUFFLES); + let mut null_p10_ap: Vec = Vec::with_capacity(SHUFFLES); + let mut null_p10_a2: Vec = Vec::with_capacity(SHUFFLES); + let mut mean_abs_pop_vs_null_rho = 0.0f32; + + for r in 0..SHUFFLES { + let seed = 0x9E37_79B9_7F4A_7C15u64 ^ (r as u64); + let shuffled_codes = fisher_yates_shuffle(&codes, seed); + + // Recompute basins with the shuffled codes. + let mut null_basin_of: HashMap> = HashMap::new(); + for &s in objects.keys() { + if objects[&s].len() < MIN_BASIN { + continue; + } + let member_codes: Vec = objects[&s] + .iter() + .map(|&o| shuffled_codes[o as usize]) + .collect(); + let b = basin_self_code(&space, s, &member_codes, &edges[&s]); + null_basin_of.insert(s, b); + } + let null_pop: Vec = candidates + .iter() + .map(|c| { + let b = null_basin_of[&c.s] + .as_ref() + .expect("filtered to Some above"); + space.distance(&shuffled_codes[c.o as usize], &b.self_code) + }) + .collect(); + let null_ap: Vec = null_pop.iter().map(|&p| -p).collect(); + let null_a2: Vec = combine_ranks(&a0, &null_ap); + + mean_abs_pop_vs_null_rho += spearman(&pop, &null_pop).abs(); + null_partials.push(partial_spearman(&null_ap, &labels, &freqs)); + null_p10_ap.push(precision_at(&null_ap, &labels, 10)); + null_p10_a2.push(precision_at(&null_a2, &labels, 10)); + } + mean_abs_pop_vs_null_rho /= SHUFFLES as f32; + + // ── guard 3: the null actually destroys the binding ──────────────── + if mean_abs_pop_vs_null_rho >= 0.5 { + panic!( + "GUARD 3 FAIL split frac={frac}: mean |spearman(pop_real, pop_null)| = \ + {mean_abs_pop_vs_null_rho:.3} >= 0.5 — the null shuffle is not actually \ + destroying the word↔code binding" + ); + } + println!( + " guard3 PASS mean|spearman(pop_real,pop_null)| = {mean_abs_pop_vs_null_rho:.3} < 0.5" + ); + + let (null_partial_mean, null_partial_p95) = mean_and_p95(&null_partials); + let (null_p10_ap_mean, null_p10_ap_p95) = mean_and_p95(&null_p10_ap); + let (null_p10_a2_mean, null_p10_a2_p95) = mean_and_p95(&null_p10_a2); + + // ── report this split ─────────────────────────────────────────────── + println!( + "\nSPLIT frac={frac:.2} cut=verse{cut} candidates={n_total} base_rate={base_rate:.3}" + ); + println!( + " {:<22} {:>7} {:>7} {:>7} {:>9}", + "arm", "p@10", "p@25", "p@100", "spearman" + ); + for a in &arms { + println!( + " {:<22} {:>7.3} {:>7.3} {:>7.3} {:>9.3}", + a.name, a.p10, a.p25, a.p100, a.spearman + ); + } + println!( + " partial_spearman(AP,label|freq): real={partial_real:.3} null_mean={null_partial_mean:.3} null_p95={null_partial_p95:.3}" + ); + + all_splits.push(SplitResult { + arms, + partial_real, + null_partial_mean, + null_partial_p95, + null_p10_ap_mean, + null_p10_ap_p95, + null_p10_a2_mean, + null_p10_a2_p95, + }); + } + + // ── guard 4: does curiosity_gestalt's magnitude ever re-order the base + // curiosity ranking, or is it always a monotone (hence rank-inert) rescale? + // Measured across ALL splits pooled, so the answer is not an artifact of + // one split's small candidate count. ── + { + let mut pooled_a0: Vec = Vec::new(); + let mut pooled_a1: Vec = Vec::new(); + let mut pooled_a1b: Vec = Vec::new(); + for &frac in SPLITS { + let cut = (max_verse as f64 * frac) as u32; + let prefix: Vec<&Row> = rows.iter().filter(|&&(v, ..)| v < cut).collect(); + use std::collections::HashMap; + let mut objects: HashMap> = HashMap::new(); + let mut edges: HashMap> = HashMap::new(); + let mut counts: HashMap<(u16, u16, u16), u32> = HashMap::new(); + for &&(_, s, p, o) in &prefix { + edges.entry(s).or_default().push((p, o)); + let objs = objects.entry(s).or_default(); + if !objs.contains(&o) { + objs.push(o); + } + *counts.entry((s, p, o)).or_insert(0) += 1; + } + let mut candidates: Vec = counts + .keys() + .filter(|&&(s, _, _)| objects.get(&s).map_or(0, Vec::len) >= MIN_BASIN) + .map(|&(s, p, o)| Candidate { s, p, o }) + .collect(); + candidates.sort_unstable(); + for c in &candidates { + let n = *counts.get(&(c.s, c.p, c.o)).unwrap() as f32; + let e = FrontierEdge { + source: vocab.word(c.s).unwrap_or("?").to_string(), + target: vocab.word(c.o).unwrap_or("?").to_string(), + label: vocab.word(c.p).unwrap_or("?").to_string(), + truth: NarsTruth { + frequency: 1.0, + confidence: n / (n + 1.0), + }, + query_count: 0, + is_seed: false, + }; + pooled_a0.push(e.curiosity()); + pooled_a1.push(e.curiosity_gestalt(&assess_a, &signals).magnitude); + pooled_a1b.push(e.curiosity_gestalt(&assess_b, &signals).magnitude); + } + } + let rho_a0_a1 = spearman(&pooled_a0, &pooled_a1); + let rho_a0_a1b = spearman(&pooled_a0, &pooled_a1b); + println!( + "\nGUARD 4 pooled over {} candidates across all splits", + pooled_a0.len() + ); + println!(" spearman(A0, A1[assessA]) = {rho_a0_a1:.6}"); + println!(" spearman(A0, A1B[assessB]) = {rho_a0_a1b:.6}"); + if rho_a0_a1 > 0.999 && rho_a0_a1b > 0.999 { + println!( + "FINDING qualia-inert: curiosity_gestalt magnitude is a per-graph scalar \ + multiple of curiosity, so it cannot reorder a frontier (measured rho = {rho_a0_a1:.6} / {rho_a0_a1b:.6})" + ); + assert!( + rho_a0_a1 > 0.999 && rho_a0_a1b > 0.999, + "guard 4 internal contradiction" + ); + } else { + println!( + "FINDING qualia-reorders: curiosity_gestalt magnitude changes the frontier \ + ordering relative to bare curiosity (measured rho = {rho_a0_a1:.6} / {rho_a0_a1b:.6})" + ); + } + } + + // ── aggregate across splits ───────────────────────────────────────────── + println!( + "\n════════════════════ AGGREGATE (mean over {} splits) ════════════════════", + all_splits.len() + ); + let n = all_splits.len() as f32; + let arm_names: Vec<&str> = all_splits[0].arms.iter().map(|a| a.name).collect(); + for (i, name) in arm_names.iter().enumerate() { + let mean_p10: f32 = all_splits.iter().map(|s| s.arms[i].p10).sum::() / n; + let mean_p25: f32 = all_splits.iter().map(|s| s.arms[i].p25).sum::() / n; + let mean_p100: f32 = all_splits.iter().map(|s| s.arms[i].p100).sum::() / n; + println!(" {name:<22} mean p@10={mean_p10:.3} p@25={mean_p25:.3} p@100={mean_p100:.3}"); + } + let mean_partial_real: f32 = all_splits.iter().map(|s| s.partial_real).sum::() / n; + let mean_null_partial_p95: f32 = all_splits.iter().map(|s| s.null_partial_p95).sum::() / n; + let mean_p10_a2: f32 = all_splits + .iter() + .map(|s| { + s.arms + .iter() + .find(|a| a.name.starts_with("A2")) + .unwrap() + .p10 + }) + .sum::() + / n; + let mean_p10_a0: f32 = all_splits + .iter() + .map(|s| { + s.arms + .iter() + .find(|a| a.name.starts_with("A0")) + .unwrap() + .p10 + }) + .sum::() + / n; + let mean_null_p10_a2_p95: f32 = all_splits.iter().map(|s| s.null_p10_a2_p95).sum::() / n; + // The null MEANS are reported alongside the 95th percentiles: a null whose + // mean sits near zero while its p95 is small is a null that genuinely + // destroyed the binding, which is the claim guard 3 makes per split. Printed + // rather than dropped -- a computed statistic that is never shown is a + // measurement the reader cannot check. + let mean_null_partial_mean: f32 = + all_splits.iter().map(|s| s.null_partial_mean).sum::() / n; + let mean_null_p10_a2_mean: f32 = all_splits.iter().map(|s| s.null_p10_a2_mean).sum::() / n; + + println!("\nmean real partial Spearman(AP,label|freq) = {mean_partial_real:.3}"); + println!("mean null partial Spearman (mean over shuffles) = {mean_null_partial_mean:.3}"); + println!("mean null partial Spearman 95th percentile = {mean_null_partial_p95:.3}"); + println!( + "mean(A2 p@10) - mean(A0 p@10) = {:.3}", + mean_p10_a2 - mean_p10_a0 + ); + println!("mean(A2 p@10) = {mean_p10_a2:.3}"); + println!("mean null A2 p@10 (mean over shuffles) = {mean_null_p10_a2_mean:.3}"); + println!("mean null A2 p@10 95th percentile = {mean_null_p10_a2_p95:.3}"); + + // The AP arm alone (population readout, no curiosity) against its own null. + let mean_p10_ap: f32 = all_splits + .iter() + .map(|s| { + s.arms + .iter() + .find(|a| a.name.starts_with("AP")) + .unwrap() + .p10 + }) + .sum::() + / n; + let mean_null_p10_ap_mean: f32 = all_splits.iter().map(|s| s.null_p10_ap_mean).sum::() / n; + let mean_null_p10_ap_p95: f32 = all_splits.iter().map(|s| s.null_p10_ap_p95).sum::() / n; + println!("mean(AP p@10) real / null-mean / null-p95 = {mean_p10_ap:.3} / {mean_null_p10_ap_mean:.3} / {mean_null_p10_ap_p95:.3}"); + + let cond_a = mean_partial_real > mean_null_partial_p95 + 0.02; + let cond_b = (mean_p10_a2 - mean_p10_a0) >= 0.05 && mean_p10_a2 > mean_null_p10_a2_p95; + if cond_a && cond_b { + println!("\nVERDICT PASS"); + } else { + println!("\nVERDICT KILL"); + } +} + +/// Precision@k: mean label over the k highest-scoring candidates. Ties broken +/// by candidate index ascending (deterministic). +fn precision_at(scores: &[f32], labels: &[f32], k: usize) -> f32 { + let mut idx: Vec = (0..scores.len()).collect(); + idx.sort_by(|&a, &b| { + scores[b] + .partial_cmp(&scores[a]) + .unwrap_or(std::cmp::Ordering::Equal) + .then(a.cmp(&b)) + }); + let k = k.min(idx.len()); + if k == 0 { + return 0.0; + } + idx[..k].iter().map(|&i| labels[i]).sum::() / k as f32 +} + +/// Normalized ascending rank in `[0, 1]`, average ranks for ties. Rank 0 = +/// smallest value. +fn nrank(v: &[f32]) -> Vec { + let n = v.len(); + let mut idx: Vec = (0..n).collect(); + idx.sort_by(|&a, &b| v[a].partial_cmp(&v[b]).unwrap_or(std::cmp::Ordering::Equal)); + let mut ranks = vec![0f32; n]; + let mut i = 0usize; + while i < n { + let mut j = i; + while j + 1 < n && v[idx[j + 1]] == v[idx[i]] { + j += 1; + } + // average rank over the tie block [i, j] + let avg_rank = (i + j) as f32 / 2.0; + for &k in &idx[i..=j] { + ranks[k] = avg_rank; + } + i = j + 1; + } + if n <= 1 { + return ranks; + } + let denom = (n - 1) as f32; + ranks.into_iter().map(|r| r / denom).collect() +} + +/// Combine two score vectors as `0.5 * (nrank(x) + nrank(y))`. +fn combine_ranks(x: &[f32], y: &[f32]) -> Vec { + let rx = nrank(x); + let ry = nrank(y); + rx.iter().zip(&ry).map(|(&a, &b)| 0.5 * (a + b)).collect() +} + +/// Spearman rank correlation (Pearson over average ranks). `0.0` if either +/// side has zero variance. +fn spearman(x: &[f32], y: &[f32]) -> f32 { + let rx = nrank(x); + let ry = nrank(y); + pearson(&rx, &ry) +} + +fn pearson(x: &[f32], y: &[f32]) -> f32 { + let n = x.len(); + if n == 0 { + return 0.0; + } + let mx = x.iter().sum::() / n as f32; + let my = y.iter().sum::() / n as f32; + let mut cov = 0f32; + let mut vx = 0f32; + let mut vy = 0f32; + for i in 0..n { + let dx = x[i] - mx; + let dy = y[i] - my; + cov += dx * dy; + vx += dx * dx; + vy += dy * dy; + } + if vx <= 0.0 || vy <= 0.0 { + return 0.0; + } + cov / (vx.sqrt() * vy.sqrt()) +} + +/// Mean and 95th percentile (index `(0.95*(len-1)).round()` of the ascending +/// sort) of a value set. +fn mean_and_p95(v: &[f32]) -> (f32, f32) { + let n = v.len(); + if n == 0 { + return (0.0, 0.0); + } + let mean = v.iter().sum::() / n as f32; + let mut sorted = v.to_vec(); + sorted.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal)); + let idx = ((0.95 * (n - 1) as f32).round() as usize).min(n - 1); + (mean, sorted[idx]) +} + +/// SplitMix64 PRNG — deterministic, no external rng crate, the workspace's +/// standard seed constant (see `bible_wave.rs::shuffle_null`). +struct SplitMix64(u64); +impl SplitMix64 { + fn next(&mut self) -> u64 { + self.0 = self.0.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = self.0; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) + } +} + +/// Deterministic Fisher-Yates shuffle of the code vector (does not mutate the +/// original; the multiset of codes is preserved exactly, only the +/// word-id↔code binding is permuted). +fn fisher_yates_shuffle(codes: &[Cam96], seed: u64) -> Vec { + let mut out = codes.to_vec(); + let mut rng = SplitMix64(seed); + for i in (1..out.len()).rev() { + let j = (rng.next() % (i as u64 + 1)) as usize; + out.swap(i, j); + } + out +}