D-TEH-3 fate probes: both KILL — semantic_chunker + spiral_segment stay LAB

.claude/board/EPIPHANIES.md

@@ -1,3 +1,21 @@
+## 2026-09-03 — E-TWO-FATE-PROBES-KILL-DIFFERENT-WAYS-1 — `semantic_chunker` fails at the mechanism, `spiral_segment` fails at the ratio; a pre-registered gate that only checked "pass/fail" would have hidden which
+
+**Status:** FINDING (measured, `crates/thinking-engine/examples/{chunker_falsifier,spiral_gate_probe}.rs`, both run on real baked data, no synthetic corpus). **Confidence:** High — both arms were pre-registered before either probe ran (plan `thinking-engine-harvest-closure-v1.md` §4b), and neither threshold was retuned after seeing a result.
+
+**What the closure plan asked.** The two remaining §1c rows each had a promotion path gated on a falsifier: `semantic_chunker` → deepnsm-v2's text side, `spiral_segment` → the codec certification battery. Both probes ran on real data (the tier-1..4 calibration corpus + the Jina v5 tokenizer + `jina-v5-codebook` for the chunker; five real baked 256x256 tables — jina-v3, bge-m3, reranker, jina-v5 u8 and i8 — for the spiral codec). Both came back KILL, and the two KILLs are structurally different failures worth telling apart.
+
+**`semantic_chunker` fails at the MECHANISM — recall = 0.000 at every threshold swept (0.30 / 0.45 / 0.60), across 168 cross-topic passages.** An all-zero result is exactly the shape a broken harness produces, so before trusting it a diagnostic re-ran the module's OWN pre-existing adversarial positive control — the synthetic maximally-separated centroid corners from its `detects_boundary_between_topics` test — against the SAME `jina-v5-codebook` table. **It also produced zero boundaries.** So the null is not a probe-design artifact: the perturb→think→top-k-Jaccard convergence pattern this module is built on simply does not diverge on an HDR-encoded table, even on inputs engineered to be as separable as the 256-centroid space allows. The module's own existing test already carried a hedge to this effect ("On uniform HDR tables the convergence patterns may not diverge strongly") — this is that hedge turned into a measured, falsifiable KILL. (The control was first run as a throwaway script and only its result recorded here; CodeRabbit's review on PR #1144 correctly flagged that a deleted diagnostic leaves the mechanism-null conclusion unreproducible, so it is now a 4th committed arm in `chunker_falsifier.rs` — same corners, same zero.)
+
+**`spiral_segment` fails at the RATIO on u8 tables (fidelity is fine there); on i8 it fails BOTH gates — corrected post-merge by a Codex review comment on this PR (`chatgpt-codex-connector[bot]`, P2), see the addendum below.** Pearson r and Spearman rho clear the 0.9980 gate on every u8 table at every max_error tried — the codec reconstructs distance rows accurately there. The i8 table does NOT clear it at any max_error tested: at its best configuration (0.005) r = 0.9993 passes but rho = 0.9975 falls short of the 0.9980 floor, and rho only gets worse as max_error loosens (0.9906 / 0.9590 / 0.7137). What kills every table, u8 or i8, is `bytes(spiral) <= bytes(u8)/2`: the best compression measured is ~0.28x on u8 (i.e. the spiral encoding is **~3.6x LARGER** than the u8 table it would replace) and ~0.80x on i8 at its loosest setting, nowhere near the module's own "51x compression" doc-comment claim or even the ecosystem's 2x floor. Root cause is visible in the segment count: a real per-row CDF from a trained codebook needs ~114-143 spiral segments (8 bytes each) to hit even a loose 0.05 max_error, because the codec's fitting premise — few segments suffice per row — only holds on smooth synthetic curves, and a trained model's distance rows are not smooth. The 51x claim was never false against the data it was tested on; it was false against the distributional shape of data it was never tested on.
+
+**Addendum 2026-09-03 (Codex review, PR #1144, verified against the probe's own re-run):** the FIRST version of this entry and the plan's §4c write-up both claimed "even the i8 table clears [fidelity] at max_error <= 0.01" — this was wrong, and the contradicting number (rho = 0.9975 < 0.9980 at max_error 0.005) was sitting in the same write-up's own results table. i8 never clears BOTH fidelity gates at any tested max_error; the KILL verdict for `spiral_segment` is unaffected (i8 also fails the compression gate at every max_error, so the module stays LAB either way), but the characterization "fidelity passes, only compression fails" is accurate for u8 only, not for i8. Corrected in the plan and here rather than silently editing the original claim away — see this repo's own falsifiability rule on not treating a code-adjacent number as verified until it's actually checked against the row that contradicts it.
+
+**The generalizable point.** "The gate failed" is not one finding — WHERE it failed changes what happens next. A chunker whose mechanism cannot discriminate needs a different algorithm entirely (or a different substrate) before it is worth re-probing; a codec whose fidelity is fine but whose ratio is wrong needs either a different fitting strategy or an honest downgrade of its claimed compression, and re-running the SAME probe on a friendlier table would not fix either — which is why neither module is being re-probed. Both fate-probe reports therefore carry a per-configuration table (not just a verdict), so a future session reading the KILL knows which failure it inherited.
+
+**Both stay LAB, not deleted, not re-probed on friendlier data** — per the pre-registration's own commitment (§4b: "Not deleted... Not re-probed on a different corpus to get a better answer"). §1c is now fully closed: every row in the harvest census table has a verdict.
+
+Refs: `thinking-engine-harvest-closure-v1` §4b (pre-registration) / §4c (results) / §5 D-TEH-3; `crates/thinking-engine/src/{semantic_chunker,spiral_segment}.rs`; `.claude/agents/falsifier-auditor.md` (the can-it-fire discipline applied to the chunker's all-zero result); the falsifiability rule (CLAUDE.md § The falsifiability rule).
+
 ## 2026-09-02 — E-THE-LIFT-GATE-FOUND-A-TIE-BLIND-SPEARMAN-1 — the D-TEH-3 comparison on a distinguishing fixture separated a same-formula copy from a wrong-estimator copy, and the tie-free fixtures the lab had used could not have
 
 **Status:** FINDING (measured, tests committed in `crates/jc/src/reliability.rs`). **Confidence:** High.

.claude/board/LATEST_STATE.md

@@ -1,3 +1,12 @@
+## 2026-09-03 — branch (D-TEH-3 closed, after #1143 merged): both fate probes KILL — INVENTORY DELTA
+
+- MERGED #1143 (D-TEH-3 math half, `f05a358`'s ancestor): `jc::drift` / `jc::quorum` land, `cronbach.rs` deleted (see the 2026-09-02 delta below).
+- ADDED (probes, not shipped API) `crates/thinking-engine/examples/chunker_falsifier.rs` — real-data falsifier for `semantic_chunker` (Jina v5 tokenizer + `jina-v5-codebook`, the tier-1..4 calibration corpus, 168 cross-topic + 8 same-topic passages, 20 SplitMix64 null permutations). Result: recall = 0.000 at every pre-registered threshold; confirmed a genuine mechanism null (not a harness bug) via a non-committed positive-control diagnostic on the module's own adversarial fixture shape. **Verdict: KILL — `semantic_chunker` stays LAB, not ported to deepnsm-v2.**
+- ADDED (probes, not shipped API) `crates/thinking-engine/examples/spiral_gate_probe.rs` — real-data falsifier for `spiral_segment` against five real baked 256x256 tables (jina-v3/bge-m3/reranker/jina-v5 u8, jina-v5 i8). Result: fidelity clears the r/rho >= 0.9980 gate on every u8 table; compression fails on every table — `ratio_vs_u8 = bytes(u8)/bytes(spiral)` tops out at ~0.28x (i.e. the spiral encoding is ~3.6x LARGER than the u8 table it would replace), not the module's claimed 51x smaller. **Verdict: KILL — `spiral_segment` stays LAB, no certification battery scheduled.**
+- FIXED (unrelated to the probes, surfaced by the same `clippy -D warnings` run and paid down per `TD-THINKING-ENGINE-EXCLUDED-DEBT-1`): `world_model.rs::from_engine_state` (10 args → `ThoughtSignals` bundle, 4 args, zero in-tree callers); `cognitive_stack.rs::MetaCognition` (`impl Default`); `dto.rs::ThoughtIndex` (`#[derive(Default)]`); `tensor_bridge.rs::pairwise_cosines` (documented `#[allow]`, symmetric double-write has no clean iterator form); `qualia.rs` (19 range-fill loops → `.fill()`, 1 zip rewrite); `signed_domino.rs` (1 enumerate rewrite). 25 of ~40 pre-existing lints paid; lib tests for every touched module green (43/43) before and after.
+- §1c of `thinking-engine-harvest-closure-v1.md` is now CLOSED — every harvested row has a verdict (ghost prior: ported; calibration math: lifted into jc; semantic_chunker: KILL; spiral_segment: KILL).
+- NOT IN THIS DELTA: the ~13 remaining pre-existing clippy lints outside `world_model.rs`/`cognitive_stack.rs`/`dto.rs`/`tensor_bridge.rs`/`qualia.rs`/`signed_domino.rs` (spread across `cognitive_trace.rs`, `contract_bridge.rs`, `f32_engine.rs`, `l4.rs`, `l4_bridge.rs`, `layered.rs`, `pooling.rs`, `branching.rs`, `composite_engine.rs` — not read this pass); the 3 unrelated `jc`-crate `dead_code` warnings (a separate crate's own debt).
+
 ## 2026-09-02 — branch (D-TEH-3, after #1142 merged): calibration math lifted into jc — INVENTORY DELTA
 
 - MERGED #1142 (D-TEH-2, `3c5f040`): no contract type; planner `nars/ghost_prior.rs` + lab-crate edits + `weather-poc.yml` sibling checkout (see PR_ARC).

.claude/board/STATUS_BOARD.md

@@ -20,7 +20,7 @@
 | D-TEH-0 | census: live footprint measured (one required consumer via `bridge_gate`; one optional via `with-engine`), 51-file fate table, open-row reconciliation, four-wave closure, idea harvest, four rulings asked | **Shipped (plan-only, this PR)** |
 | D-TEH-1 | W1: `bridge_gate` (seven items) → `lance_graph_contract::bridge_gate`; callcenter re-imports and drops the path dep; thinking-engine keeps a re-export shim | **Shipped 2026-09-02** (before: required dep, 6 crossing sites, dep-drop alone fails 6 × E0433; after: zero thinking-engine deps in callcenter metadata, contract 1303/1303, callcenter 156/156, driver default + `with-engine` green, ALU artery files byte-identical). `with-engine` re-point deferred by stop condition: D-TTV-1 not landed |
 | D-TEH-2 | W2: ghost prior → planner `nars/ghost_prior.rs` over `WisdomMarker`, per-thought, two-sided falsifiers; crate `ghosts.rs` deleted | **Shipped 2026-09-02** — `GhostPrior`/`PriorFloor` in planner `nars/ghost_prior.rs`, 14 tests incl. the calibration gate (default floor = `Marker`, the gate reversed the first declaration); `ghosts.rs` + `think.rs` deleted; TD-GHOST-ECHO-DUP-1 resolved; consumer D-HOUSE-4 unblocked |
-| D-TEH-3 | W2: calibration MATH → `jc` (ruling 4, 2026-09-02: lift if correct, perfect in jc if not; crate copies die); glue stays in the lab crate; `semantic_chunker` / `spiral_segment` decided by their falsifiers | **Math half Shipped 2026-09-02** — `jc::drift` (re-encode drift + delta summary) and `jc::quorum` (pairwise agreement, bands, Cronbach report) added; cronbach LIFTED (same formula; `f32` copy loses the `1e7`-shifted fixture), spearman PERFECTED-IN-JC (retired copy was tie-blind: 1.000 vs 0.9487 on one tie); lab `cronbach.rs` deleted, `reencode_safety` / `silu_correction` / `ground_truth` are glue over jc. Open: the `semantic_chunker` / `spiral_segment` falsifier halves |
+| D-TEH-3 | W2: calibration MATH → `jc` (ruling 4, 2026-09-02: lift if correct, perfect in jc if not; crate copies die); glue stays in the lab crate; `semantic_chunker` / `spiral_segment` decided by their falsifiers | **Shipped 2026-09-02/03, all three halves closed** — `jc::drift` (re-encode drift + delta summary) and `jc::quorum` (pairwise agreement, bands, Cronbach report) added; cronbach LIFTED (same formula; `f32` copy loses the `1e7`-shifted fixture), spearman PERFECTED-IN-JC (retired copy was tie-blind: 1.000 vs 0.9487 on one tie); lab `cronbach.rs` deleted, `reencode_safety` / `silu_correction` / `ground_truth` are glue over jc. `semantic_chunker`: **KILL** — recall 0.000 at every pre-registered threshold, confirmed a real mechanism null via a committed positive-control arm in `chunker_falsifier.rs` (the module's own adversarial synthetic-corners shape, added 2026-09-03 per CodeRabbit review on #1144 so the evidence is reproducible, not just asserted); stays LAB. `spiral_segment`: **KILL** — fidelity passes on every u8 table (r/rho >= 0.998) but NOT on i8 (rho tops out at 0.9975, below the 0.998 floor, corrected 2026-09-03 per Codex review on #1144); compression fails on every table regardless — `ratio_vs_u8 = bytes(u8)/bytes(spiral)` tops out at ~0.28x (i.e. the spiral encoding is ~3.6x LARGER than the u8 table it would replace), not the claimed 51x smaller; stays LAB, no certification battery scheduled. Plan `thinking-engine-harvest-closure-v1.md` §4c |
 | D-TEH-4 | W3: ENTROPY M8 engine collapse with dtype parity suite; 5 cascade shapes + 3 lens modules collapse | Queued |
 | D-TEH-5 | W4: residue deleted, crate renamed `thinking-lab` with a `--manifest-path` CI line; §2 rows closed; `TD-THINKING-ENGINE-EXCLUDED-DEBT-1` paid | Queued — closes the chapter |
 

.claude/board/TECH_DEBT.md

@@ -991,6 +991,8 @@ Both recorded so P4 (D-TRI-2/3 ancestry-pipeline consolidation) inherits them. R
 
 **Surfaced during P3** (rung dedup). thinking-engine is a workspace-EXCLUDED crate (root Cargo.toml `exclude`), so no CI/clippy gate runs on it — it has accumulated ~40 `clippy -D warnings` lints (loop-index, `map_or` simplify, too-many-args, missing-Default; all in modules P3 never touched — e.g. `cognitive_stack.rs:242` MetaCognition::new, `world_model.rs:154` from_engine_state) and at least one test-compile break (`DualResult` missing `convergence_signed`/`convergence_unsigned` fields, a stale test). The **lib builds clean**; only `--tests` and `clippy -D warnings` fail. Correlates with E-RUNG-ASCENT-WIRED-1's finding that the whole crate is orphaned from the production spine (only bridge_gate→callcenter + dto→driver[feature] are external wires). PAYOFF: gated on the P4 ancestry-consolidation decision (wire the gems into the spine → then it earns a CI slot and the debt gets paid; or retire the orphaned cluster). Not paid in P3 (out of scope; excluded crate).
 
+**Update 2026-09-03 (D-TEH-3 fate-probe PR, partial paydown, both named items resolved).** Both examples the original entry NAMED explicitly are fixed: `world_model.rs:154` (`from_engine_state`, 10 args → grouped into a new `ThoughtSignals` bundle, 4 args; zero in-tree callers so behaviour is unchanged by construction, verified by grep across the whole repo) and `cognitive_stack.rs:242` (`MetaCognition::new` → added `impl Default for MetaCognition` calling `new()`, standard `new_without_default` fix). Also paid down while touching adjacent files surfaced by the same `clippy -D warnings` run: `tensor_bridge.rs::pairwise_cosines` (scoped `#[allow(clippy::needless_range_loop)]` with a comment — the symmetric double-write to `matrix[i][j]`/`matrix[j][i]` alongside `self.embeddings[i]`/`[j]` has no single-iterator expression, clippy's own suggested rewrite only reaches row `i`), `qualia.rs` (19 `for i in A..=B { w[i] = C; }` range-fill loops in `family_band_weights` → `w[A..=B].fill(C)`, plus `to_voice_channels`'s zip-instead-of-index rewrite — 20 lints, all mechanical constant-fills, zero behaviour risk), `signed_domino.rs` (one `for j in 0..n` → `for (j, &val) in row.iter().enumerate()`), `dto.rs::ThoughtIndex` (`#[derive(Default)]` — every field is a `Vec`, whose default is empty, matching `new()`'s body exactly). **25 of the ~40 lints paid; `cargo test --lib` on every touched module green (43/43) before and after.** Remaining ~13 (excluding 3 unrelated `jc`-crate dead_code warnings that are a separate crate's own debt) span `cognitive_trace.rs`, `contract_bridge.rs`, `f32_engine.rs`, `l4.rs` (x3), `l4_bridge.rs` (x2), `layered.rs`, `pooling.rs`, `branching.rs`, `composite_engine.rs` — not read this pass, left as this entry's continuing debt rather than risking a behaviour change on unfamiliar code under a Sonnet-tier grindwork budget. The P4 ancestry-consolidation gate this entry's PAYOFF names is still the real close.
+
 ## TD-ONTOLOGY-CLIPPY-DEBT-1 — lance-graph-ontology (member) has pre-existing clippy -D warnings errors that block dependent `-p` clippy (2026-07-17)
 
 **Surfaced during P3.** `cargo clippy -p cognitive-shader-driver -- -D warnings` fails (exit 101) entirely inside `lance-graph-ontology` (a transitive dep): `doc-lazy-continuation` ×N, `iter-cloned-collect` (`.iter().copied().collect()` → `to_vec()`), and deprecated `oxrdf::Subject` type-alias uses (ttl_parse.rs, owl.rs, op_emitter.rs, lib.rs). P3 touches none of ontology; the driver's OWN code is clippy-clean (0 findings in driver.rs). Pre-existing by construction (not in P3's diff). PAYOFF: mechanical (doc reflow + `to_vec()` + `NamedOrBlankNode` migration), its own small PR — not P3 scope. Flags that repo CI likely does not run full-workspace `clippy -D warnings`, or main is already red there.