diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 566441a92..5526d7d98 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,396 @@ +## 2026-08-20 — E-DISMECH-CORPUS-CENSUS-1 — the DisMech corpus measured: 87.2 MB of strings, of which the entire causal semantics is bits + codebook ordinals + +**Status:** FINDING (measured on upstream `monarch-initiative/dismech`, 2,100 +disorder files, fetched ephemeral to `/tmp` — never committed, per the +`/tmp`-fixture rule). **Confidence:** High — every number is a count. + +**Total string bytes across all properties: 87,222,222.** The top ~27 +properties by volume are free text (descriptions / snippets / explanations / +reference titles). Everything that carries CAUSAL SEMANTICS is tiny. + +**`causal_link_type` is EXACTLY the four `CausalTopology` states** — measured, +not assumed, so the CE64 bits 59+60 mapping is source-authoritative and needs +no inference from confidence, edge count, or predicate name: + +| state | count | +|---|---| +| DIRECT | 9,073 | +| INDIRECT_UNKNOWN_INTERMEDIATES | 4,539 | +| INDIRECT_KNOWN_INTERMEDIATES | 3,978 | +| UNKNOWN | 408 | +| **total causal edges** | **17,998** | + +**The two experimental populations this hands us (better than hiding paths):** +`IndirectKnownIntermediates` (3,978) is the ORACLE population — the source +names the mediators, so they can be hidden and recovery measured. +`IndirectUnknownIntermediates` (4,539) is the RESTRAINT CONTROL — the source +itself does not know, so a reasoner that "recovers" a mediator there is +hallucinating closure. Success is therefore two-sided: recovery sensitivity +AND epistemic restraint. + +**Every evidence enum is bits:** `supports` 4 (2 b), `evidence_source` 5 (3 b), +`modifier` 7 (3 b), `frequency` 19 (5 b), `treatment_effect` 5, +`genetic[].relationship_type` 10, `prevalence[].measure_type` 8. +⚠ `phenotypes[].category` is **261 distinct — OVER the 255 `Codebook` cap**, +and its top values include both `Neurologic` AND `Neurological`: lexical noise +inside an "enum". It needs normalization or a deliberate split, never a silent +widening (`Codebook::intern` returning `None` IS the split signal). + +**Bibliography — LLM-generated, so identity must not be the title.** 131,904 +reference-title occurrences over 31,361 distinct (4.21x reuse); 12,124,736 B +inline -> 3,051,438 B deduped (74.8% saved); with a u32 key per occurrence, +3,579,054 B total = 3.4x smaller. A stable key ALREADY exists on ~104,700 +occurrences: PMID (dominant), DOI, ORPHA, CGGV, ClinicalTrials, URL. So the +key is `(namespace, id)` — never a hash of the title, because wording drifts +between regenerations while the citation does not. + +**"MONDO-derived" applies to the DISORDERS, not the EDGE ENDPOINTS.** Measured +prefix distribution over edge endpoints: no prefix at all **25.2%**, HP 24.9%, +GO 11.6%, NCIT 10.5%, hgnc 7.6%, CL 7.3%, **MONDO only 4.4%**, UBERON 3.3%, +CHEBI 3.1%, NCBITaxon/ECTO/RO 2.0%. OBO_CORE's five namespaces cover **32.7%**. +This is the single largest correction to the DisMech-overlay plan: grounding +cannot lean on MONDO. + +**Deterministic resolution ladder — no LLM used.** Phenotypes carry NO CURIE at +all (25,120 entries, 0.0%), so the mapping must be MADE. Against the real +HP/MONDO/UBERON/PATO labels: + +| population | exact | +case/punct | +singular | RESOLVED | unresolved | +|---|---|---|---|---|---| +| phenotype labels (25,120) | 55.3% | 23.2% | 2.1% | **80.7%** | 19.3% (3,647 distinct) | +| unprefixed endpoints (30,872) | 25.6% | 9.9% | 0.9% | **36.4%** | 63.6% (11,385 distinct) | + +**The unresolved tails are three DIFFERENT kinds, not one backlog:** +(a) provenance leaking into the endpoint slot — `Orphanet` (120), `OMIM` (32), +`ClinGen` (23) are database names, not concepts; (b) qualified mechanism +PROPOSITIONS — `Impaired Terminal Electron Transfer and ATP Synthesis` (24), +`Impaired Neurodevelopment` (20) — genuinely DisMech-local, and must not be +bullied into an ontology node; (c) lexical variants of real concepts. Note +`Sensorineural Hearing Loss` and `Sensorineural hearing loss` BOTH fail, so +they are a SYNONYM miss (HPO's own label is "Sensorineural hearing +impairment"), not a case miss — the next deterministic rung is the synonym +table, before any model is invoked. + +**⚠ PHENOTYPES NEED THEIR OWN RESOLUTION DOMAIN (operator, confirmed by +measurement).** Collapsing HP with MONDO does not merely strip edges — it +MISROUTES them, which is harder to detect. Measured: **1,169 labels exist in +more than one namespace, 1,129 of them HP+MONDO**. Of 19,738 resolved +phenotype labels, **26.2% landed in MONDO rather than HP**, and **23.7% +(4,678) are genuinely ambiguous** — a collapsed resolver picks by insertion +order, silently reattaching a phenotype edge to a same-named disease node. +HP must be scoped FIRST, with any cross-namespace fallback deliberate. + +**UBERON is the capstone, measurably:** 10 collisions across 14,975 UBERON +labels = **0.07%**. Anatomy is near-collision-free exactly where phenotype and +disease are not, so it is the one layer safe to anchor against. + +> **⊘ ARCHITECTURAL CONCLUSION CORRECTED (2026-08-20, same day, measured).** +> **The MEASUREMENTS above stand unchanged** — 1,169 multi-namespace labels, +> 26.2% mis-landing, 23.7% ambiguous, UBERON at 0.07%. What is corrected is +> the word **"OWN"**: the sanctioned abstraction already exists and this entry +> did not know it. +> +> `medcare-cohorts/src/quad_tenant.rs` ships `Domain` (8 variants) with a +> classid→domain map, an enforced two-witness contract (classid AND TUI must +> agree), and `FacetRegime::PerRowTui` for the CUI horseshoe: +> +> | Domain | vocabularies mapped | +> |---|---| +> | `Domain::Phenomenology` | HP | +> | `Domain::Anatomy` | **UBERON + FMA** | +> | `Domain::Disease` | MONDO + ICD-10-GM + ORDO + OMIM + DECIPHER | +> | `Domain::Lab` / `Substance` / `Procedure` | LOINC / ATC+RxNorm+Gelbe Liste / OPS | +> | `Imaging`, `BiologicalProcess` | declared, no vocabulary yet | +> +> So the corrected statements are: +> +> - *Phenotype resolution must be constrained to the existing +> `Domain::Phenomenology`; **HP is its currently populated single-facet +> vocabulary, not a new resolution domain.*** +> - *Anatomy resolution targets `Domain::Anatomy` (UBERON **and** FMA); +> UBERON's 0.07% collision rate makes it a strong current +> projection/anchor, **not the semantic domain itself**.* +> +> **Why this pedantry earns its place:** the board is executable archaeology. +> Left as written, "HP needs its own resolution domain" is precisely how a +> later session mints an `HpResolutionDomain` beside the already-built +> `Domain::Phenomenology` — the duplicate-vocabulary failure this workspace +> has now recorded several times. Append-only: the original claim is regraded +> here, not deleted. + +--- + +## 2026-08-20 — I-STRINGS-ARE-CAM-INDEX-ONLY-1 — strings in the hot path resolve through a codebook; the ONLY string home is the CAM index codebook; NEVER in any SoA + +**Status:** OPERATOR RULING (verbatim: *"All strings in hot path are mandatory to +use codebooks, the only occurrence of strings is in content addressable memory +index codebook / Never in any SoA"*; sharpened same message: *"In hot path, only +patient data or KV Side Car table"*). **Confidence:** High — iron rule. + +**The rule.** In the hot path a semantic identity is an ORDINAL. A `String` may +exist in exactly two places: (a) the **CAM index codebook** — the +content-addressed cold store / family codebook that maps bytes or a label to its +ordinal; (b) a **KV side-car table** or patient data. A string may NEVER live in +an SoA row, a value tenant, an overlay, or any reasoning-path struct. + +**The sanctioned homes already exist — this rule is not a request for new +machinery, it is a demand to stop bypassing what is built:** +- `content_store.rs` — content-addressed cold text/blob store, `ContentAddress` + = fnv1a-64 of the stored bytes, write side membrane-only. This IS the CAM index. +- `codebook.rs` — `family -> Codebook`, <=255 entries, 1-byte in-family index, + index 0 reserved as the `EdgeBlock` empty-slot sentinel; a family that + outgrows 255 SPLITS rather than widening the byte. + +**Measured compliance in this repo (2026-08-20 audit).** + +COMPLIANT, do not touch: `nars/belief.rs` carries ZERO strings — `CStmt { s: u16, +cop: Copula, p: u16 }`, `Copula::Rel(u16)`; `stance.rs:51 Interner +{ map: HashMap, names: Vec }` is the correct string->ordinal +membrane feeding it; `recipes.rs` `code`/`name`/`substrate` are `&'static str` +catalogue metadata reached by `id: u8`, never used for dispatch or equality. + +VIOLATIONS: +- `literal_graph.rs:72` `label_codebook: Vec` — an ad-hoc codebook + duplicating `codebook::Codebook`'s exact shape. Worse: `ensure_label` COMPUTES + the ordinal on every `add_node`/`add_edge` and then DISCARDS it, keeping only + the `String` on the node/edge. +- `literal_graph.rs:21,25,39,41,43` — `id`/`label`/`source`/`target` as `String` + IDENTITY, plus three `HashMap` adjacency indices. +- `exploration.rs:140-142` `FrontierEdge.source/target/label: String` — the LIVE + ranking substrate (`curiosity()` + MUL-weighted next-edge choice). +- `exploration.rs:309-311` `ExplorationResult.confirmed/denied: + Vec<(String,String,String,NarsTruth)>` — an SPO triple driving NARS revision + as three heap Strings, in a repo whose NARS statement type is three `u16`s. +- Additional interners beside the sanctioned two: `deepnsm/vocabulary.rs:82` + `forms: HashMap`; `lance-graph-cognitive/fabric/gel.rs:97` + `labels: HashMap`. + +**Predicate ordinals are TWO bytes, not one.** `ogar-loco/src/lib.rs:347` sets +`DOMAIN_FLOOR = 0x90`: `0x00..=0x8F` is universal ABI forever, `0x90..=0xFF` is +DOMAIN-LOCAL. `ogar-dismech` mints `0x90..=0xA2` (19, test-pinned) and `ogar-ro` +mints `0x90..=0xA5` (22) — the SAME range. A bare `FnIndex` is therefore +ambiguous; predicate identity is `(vocabulary, FnIndex)`, or the vocabulary is +implied by the lane's classid exactly as `ogar-obo` already does it. + +**Correction recorded with the rule:** an audit pass this session reported +`ClassRowSchema`/`RowField`/`ValueSchema` as ABSENT. They exist — +`ogar-obo/src/layout.rs:36,61` and `lance_graph_contract::canonical_node::ValueSchema`. +What IS absent is narrower: a slot->ROLE mapping (subject/predicate/mediator). +`ClassRowSchema` carves FIELDS (`entity_type`, `edge_lanes`), not roles. + +**Consequence for the reasoning overlay.** An overlay stores an ordinal, a slot +position and state bits — never a label, a relation name, or a path. Slot +position and schema ARE information; re-encoding them as prose is the redundancy +this rule exists to kill. + +--- + +## 2026-08-20 — E-S3-0-NEEDED-NO-NEW-ADDRESS-1 — the Stage-3 "S3.0 address" slot is closed as NOT-NEEDED; `IdentityQuad` already carries an exact four-component identity, at u24, inside the sanctioned V3 facet + +**Status:** FINDING (operator-directed audit; measured against merged code). +**Confidence:** High — every claim below is a read of shipped source or a +merged measurement, not a derivation. + +**The question that closes it** (operator's rule, stated during this session): +*"WHAT EXACT INFORMATION CANNOT BE EXPRESSED BY THE ADDRESSING THAT ALREADY +EXISTS?"* — and if there is no concrete falsifier showing existing addressing +insufficient, **no new absolute-address type is minted.** + +**Answer: nothing that could be demonstrated.** `identity_quad::IdentityQuad` +(operator-RATIFIED 2026-08-17, `ISS-IDENTITY-QUAD-WIDE-CARVING-HOME`) already +materializes **four external identity spaces as `4 × u24` contiguous in ONE +96-bit V3 facet payload** behind a `classid(4)`, via +`LegacyOutlier::WideTriple`. It refuses rather than truncates +(`QuadError::OrdinalTooLarge`, `MAX_ORDINAL = 2^24 − 2`); its codebooks refuse +rather than saturate (`CodebookError::TooLarge`). Its stated purpose is to +resolve a crosswalk ONCE at bake time so a read becomes a fixed-offset register +read — no join, no crosswalk walk. + +**A proposed `4 × u16` literal type was WITHDRAWN, on two independent grounds:** + +1. **`u16` cannot hold a real ontology identity.** MedCare-rs + `docs/ONTOLOGY_BAKE_STATE.md`:182 states it directly — *"real OBO ids run + past `u16` (MONDO:0700092 = 700,092) and one V3 field cannot hold that."* + The substrate already solved this with the V3 rail + (`family:identity = (num >> 16, num & 0xFFFF)`, read via + `obo_store::row_addr`). A u16 subject would have silently mis-addressed the + largest ontology in the bake. Calling such a tuple "absolute identity" was + an overclaim. +2. **`CausalLiteral` was the wrong universality.** Its own test asserted + `TREATED_WITH` — proving the structure is GENERIC. `ASSOCIATED_WITH` / + `TREATED_WITH` / `CAUSES` / `MEDIATES` / `PART_OF` are exact predicate + identities over one generic literal substrate. **Causality is a predicate + family / qualification, never universal identity.** Had a primitive been + needed it would have been `ExactLiteralAddr(D,S,P,O)` — but per the rule + above, none was. + +**Sibling absolute-address surfaces already merged**, for a future session's +map: `ogar_elk::ClassAddr` (`classid: u32 + identity: u32`, explicitly a +pre-bake **join key**, *"not an ABI address, and deliberately not documented as +one"*), `canonical_node::NodeGuid` + the HHTL cascade, and the V3 rail above. + +**The genuinely open addressing gap is a DIFFERENT one, and is not fixed by a +literal type:** `ClassId = u16` (`class_view.rs:54`) is near-exhausted for +RELATIONS — MedCare-rs `CLAUDE.md` commitment #10: *"cannot address a relation +— 11 prefixes, 8 of 280 ids over the ceiling"*, echoed in +`RAIL_OFFENE_POSTEN.md`. That is a **classid-mint capacity** question owned by +OGAR/lance-graph, to be raised with the operator in session. + +**What the ladder says the real work is** (operator-requested matrix): + +| | ADDRESS | HYDRATED SoA | TRAVERSAL | +|---|---|---|---| +| Bible / Rosetta | yes | **NO** | partial / context | +| OSM | yes | yes | overlay / junction | +| MedCare ontology | yes | yes | **YES, Stage 1** | +| DisMech oracle | source | structured | causal oracle | + +**The empty column is not ADDRESS.** The missing work is *hydrate epistemic / +causal nodes → reason over them → think about the reasoning*. Next target is +the DisMech oracle experiment: hide known mechanism intermediates, hydrate the +addressed ontology neighbourhood, let NARS/recipes recover candidates, compare +against DisMech truth (`dismech-rs` `graph::build_causal_graph`, falsified at +1,995 diseases / 33,458 edges; 1,903 committed `pathographs/MONDO_*.json` as +ground truth). + +**Also withdrawn with the type: `routing_prefix()`.** Concatenating D/S/P/O +nibbles into a `NiblePath` yields deterministic **lexicographic** prefixes. It +was labelled an "HHTL locality / cohort projection" with no consumer and no +measurement establishing that it preserves HHTL *semantic* locality — +`NiblePath` is built for the `subClassOf` Abstammung tree, where a prefix is an +ancestry claim; a lexicographic prefix over concatenated ordinals carries no +such guarantee. If it returns it needs a real consumer and a measurement first. + +**Cross-ref:** `E-NIBLEPATH-DEPTH-IS-NOT-HHTL-DIMENSIONALITY-1` (below), +`E-WORDNET-IS-A-LOCALITY-PRIOR-NOT-AN-IDENTITY-ENCODING-1` (below), PR #973 +(closed unmerged), `.claude/handovers/2026-08-20-s3-0-cold-start-recovery-audit.md`. + +--- + +## 2026-08-20 — E-WORDNET-IS-A-LOCALITY-PRIOR-NOT-AN-IDENTITY-ENCODING-1 — #875 measured a taxonomy-informed HHTL *search prior*; it did NOT prove an injective WordNet address, and a session citing it as one was corrected + +**Status:** CORRECTION (operator-caught, 2026-08-20). **Confidence:** High — +#875's own W5 numbers settle it. + +**The overclaim.** A recovery session (this one) cited +`E-WORDNET-MAKES-THE-4-ARY-ADDRESS-SEMANTIC-1` as proving *"a full-width 4-ary +HHTL fold of real WordNet ancestry is an EXACT structural encoding"*, and used +that as a counterexample licensing exact HHTL literal identity. **False.** + +**#875's own W5 gate says so:** *256/256 cells used, occupancy min 29 / median +255 / max 1270* over **65,292 addressed leaves**. That is ~255 leaves per cell. +The fold is **many-to-one by a factor of hundreds** — not injective, not an +identity encoding, and never claimed to be one in #875's own text. + +**What #875 DID measure**, and it is a strong result on its own terms: a +**deterministic, taxonomy-informed HHTL locality / search prior.** +corr(shared address levels, LCA depth) **+0.494** real vs **−0.036** shuffled; +out-of-cell band recall **0.763 vs 0.031 random = 24.71×**; a **2.47-hop** +sub-nibble distinction the 16-ary router cannot address. Explicitly a +*discriminating* prior that does NOT cover everything — #875 dropped its own +cover guard as inert for exactly that reason, and its Boundaries section +already warns that *"a concept with two genuine parents is representable at +only one address"* (first-`@`-parent tree projection of a DAG). + +**Therefore the Bible/Rosetta + WordNet composition reads:** +`frozen verse identity × quasi-absolute taxonomy-informed semantic coordinate × +witness / qualia planes` — **not** "256 HHTL cells uniquely identify a lexicon." + +**The transferable lesson, and it is the same shape as the one below.** The +retraction of #973 was correct; the replacement reached for the nearest +merged-and-measured result to license the opposite conclusion and **inflated +what that result said** in the process. A counterexample cited past its own +measured Boundaries section is not a counterexample — it is the original +failure mode with the sign flipped. Read the gate table, not the headline. + +**Cross-ref:** `E-WORDNET-MAKES-THE-4-ARY-ADDRESS-SEMANTIC-1` (unchanged; its +own Boundaries section was always correct), `E-S3-0-NEEDED-NO-NEW-ADDRESS-1`. + +--- + +## 2026-08-20 — E-NIBLEPATH-DEPTH-IS-NOT-HHTL-DIMENSIONALITY-1 — retracts #973's `E-THE-LITERAL-CANNOT-LIVE-IN-THE-PATH-IT-ROOTS-1` + +**Status:** FINDING (retraction; PR #973 closed unmerged, so the retracted +entry never landed on `main` — recorded here so the reasoning is not repeated). +**Confidence:** High. + +**The retracted claim.** #973 measured, correctly, that a +`domain·subject·predicate·object` at `u16` each is exactly 16 nibbles — the +entire budget of `hhtl::NiblePath`, a single sequential `u64`-backed router +path built for the `subClassOf` Abstammung tree. From that true LOCAL fact it +concluded, under a title that reads as a global claim, that exact identity +cannot live in HHTL addressing and evidence must **ref-escape** out of address +space into a structurally separate mechanism. + +**Why the inference is not licensed.** `MAX_DEPTH` is a property of ONE +depth-limited single-path router, not of HHTL addressing as a concept. The +substrate carries other HHTL-adjacent shapes that are not sequential descent: +`facet::FacetCascade` is a fixed 16-byte register supporting SEVERAL +SIMULTANEOUS ClassView-selected readings of the same bytes +(`G3D4`/`G4D3`/`G6D2`/`24×i4`), with `tekamolo_facet` as the shipped instance — +four orthogonal 256:256:256 lanes over one register, not nested depth. And +`E-ANAPHORA-BEYOND-I4-IS-A-BASIN-EDGE-1` (7,657 real German relative clauses, +88.01% in-window) is the general pattern: a local representation exhausting +cleanly marks a TYPE boundary — switch to another sanctioned reading of the +same address, never widen the pointer or declare the substrate exhausted. + +**What survives.** The arithmetic, scoped: `4 × u16 == 16 nibbles == +NiblePath::MAX_DEPTH`, and such a path is `is_full()` for that router. + +**What does NOT follow from it**, and was separately settled by +`E-S3-0-NEEDED-NO-NEW-ADDRESS-1` above: that a new address type was needed at +all. It was not. + +--- + +## 2026-08-20 — E-A-LOCAL-DERIVATION-CANNOT-OVERRULE-A-MEASURED-COUNTEREXAMPLE-1 — and its twin: a counterexample cited past its own Boundaries section is the same failure with the sign flipped + +**Status:** RULING (operator, cold-start recovery session after #973's +closure). **Confidence:** High — process rule, with two instances measured in +ONE session. + +**Instance 1 (#973).** A locally correct derivation about one representation +(`NiblePath`'s 16-nibble ceiling) was promoted, via a dramatic finding title, +into an implied GLOBAL substrate conclusion — while three already-measured +counterexamples in the same repository showed the opposite pattern working. +None were consulted before the finding was written. + +**Instance 2 (the recovery PR itself, caught by the operator).** The session +correcting instance 1 then (a) cited #875 as proving an *exact* WordNet +encoding when its own W5 gate reports ~255 leaves per cell, (b) called a +`4 × u16` tuple "absolute identity" when a merged doc states plainly that real +OBO ids exceed `u16`, (c) named a generic `(D,S,P,O)` structure `CausalLiteral` +while its own test used `TREATED_WITH`, and (d) called a lexicographic prefix an +"HHTL locality projection" with no consumer and no measurement. **Four unearned +claims inside the PR whose entire purpose was retracting one.** + +**The rules that follow, for this and every future session:** + +1. Before declaring the substrate "cannot" do something architectural, search + `EPIPHANIES.md` and the relevant `.claude/knowledge/` / `docs/` for existing + measurements FIRST. A local proof whose premise omits established substrate + behaviour is not a discovery. +2. **A counterexample must be read to its Boundaries section, not its + headline.** Citing a measured result past what it measured is the same + error, inverted — and it is *easier* to commit while correcting someone + else, because the counterexample feels like it is on your side. +3. Before minting any new absolute-address type, answer in writing: *what exact + information cannot be expressed by the addressing that already exists?* No + concrete falsifier ⇒ no new type. A slot in a plan is not a falsifier. +4. Name a type for what it structurally IS, not for the first use case that + motivated it. If a test can substitute a non-causal predicate and the type + still works, "Causal" does not belong in the name. +5. Use explicit `[MERGED]` / `[MEASURED]` / `[RULING]` / `[PROPOSED]` / + `[REJECTED]` labels, and never blur "true of one type" into "true of the + substrate" without saying so in the same sentence. + +**Cross-ref:** `E-S3-0-NEEDED-NO-NEW-ADDRESS-1`, +`E-WORDNET-IS-A-LOCALITY-PRIOR-NOT-AN-IDENTITY-ENCODING-1`, +`E-NIBLEPATH-DEPTH-IS-NOT-HHTL-DIMENSIONALITY-1`, PR #973 (closed unmerged), +`.claude/handovers/2026-08-20-s3-0-cold-start-recovery-audit.md`. + +--- + ## 2026-08-20 — E-THE-COMPAT-ENUM-WAS-EATING-HALF-THE-REGISTER-1 **Status:** FINDING (measured + fixed, PR #971). `CausalEdgeV3::rehydrate` diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index 946b7600e..5808e50a2 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -1,3 +1,115 @@ +## 2026-08-20 — branch `claude/lance-graph-stage-3-recovery-2wrdbd` — DisMech compact evidence vocabulary + citation sidecar + +### Current Contract Inventory — 1 new zero-dep module + +- **`lance_graph_contract::dismech_evidence`** (new): + - `DismechTopology` — the four measured `causal_link_type` states, 2 bits, + `from_source`/`as_source`/`to_bits_2`/`from_bits_2`, plus + `source_knows_intermediates()` / `mediator_unresolved()` which separate the + 3,978-edge ORACLE population from the 4,539-edge RESTRAINT CONTROL. + - `Supports` (4, 2 b), `EvidenceSource` (5, 3 b) — round-tripped exhaustively. + - `CitationNamespace` (PMID/DOI/ORPHA/NCT/CGGV/URL), `CitationKey` + (`Identified{namespace,id}` | `ContentAddressed(ContentId)`), + `BibliographyRecord{key, title: ContentId}`. +- **SOURCE-SIDE ONLY — deliberately does NOT reference `CausalEdge64`.** The + durable causal overlay must not become a pile of hot reasoning registers + (operator ruling); `DismechTopology -> CE64 bits 59..60` happens at + HYDRATION, in the consumer, as a 1:1 read of `to_bits_2()`. +- **Every parse FAILS CLOSED.** `UNKNOWN` is a value the corpus asserts 408 + times, so minting it from a parse failure would forge an assertion the source + never made — pinned by + `unrecognised_topology_fails_closed_and_never_becomes_unknown`. +- **Citation identity never derives from the title** — pinned by + `reference_identity_survives_a_title_rewording`, the falsifier that matters + for an LLM-generated corpus. Where no stable identifier exists, + `CitationKey::ContentAddressed` says so explicitly rather than synthesising a + bibliographic id. + +### Gates + +`lance-graph-contract` **1178/1178** (7 new); `cargo fmt` clean; `cargo clippy +--all-targets --no-deps -D warnings` clean. + +### NOT frozen by this PR (measurement says do not) + +The endpoint codebook and the DisMech-local identity scheme stay OPEN until the +unprefixed population is probed — 63.6% of unprefixed endpoints are still +unresolved and the tail contains three different KINDS (provenance leakage, +mechanism propositions, lexical variants). Full numbers: +`E-DISMECH-CORPUS-CENSUS-1`. Also open: phenotype resolution must be scoped to the EXISTING `Domain::Phenomenology` (HP is its populated vocabulary, not a new domain — see the ⊘ correction on that entry) +(23.7% of resolved phenotype labels are HP/MONDO-ambiguous), and +`phenotypes[].category` at 261 distinct exceeds the 255 `Codebook` cap. + +--- +## 2026-08-20 — branch `claude/lance-graph-stage-3-recovery-2wrdbd` — S3.0 CLOSED AS NOT-NEEDED (no new type); #973 retraction + two overclaim corrections + +### Contract Inventory — NO CHANGE + +**This PR adds no type, no module, no bit, no tenant, no layout version.** It +is a retraction + audit. `crates/lance-graph-contract/src/lib.rs` is byte-clean +against `main`. + +### What was withdrawn, and why + +A first draft of this recovery PR reintroduced #973's `CausalLiteral` +(`4 × u16`, 8 bytes) with corrected prose. **The operator stopped it before +merge, and was right on four counts** — all now recorded as +`E-A-LOCAL-DERIVATION-CANNOT-OVERRULE-A-MEASURED-COUNTEREXAMPLE-1` instance 2: + +1. **Wrong universality.** Its own test asserted `TREATED_WITH` — the structure + is GENERIC. Causality is a predicate family / qualification, never universal + identity. +2. **`4 × u16` is not absolute.** MedCare-rs `ONTOLOGY_BAKE_STATE.md`:182: + *"real OBO ids run past `u16` (MONDO:0700092 = 700,092)."* The V3 rail + already solves this. +3. **A WordNet overclaim.** #875 was cited as an "EXACT structural encoding"; its + own W5 gate reports **256 cells, occupancy median 255, over 65,292 leaves** — + a locality/search prior, never an identity encoding. +4. **`routing_prefix()` was unearned.** Lexicographic prefix over concatenated + ordinals, labelled an "HHTL locality projection", with no consumer and no + measurement. + +### The question that closed the slot + +> **WHAT EXACT INFORMATION CANNOT BE EXPRESSED BY THE ADDRESSING THAT ALREADY +> EXISTS?** + +**Nothing demonstrable.** `identity_quad::IdentityQuad` (operator-RATIFIED +2026-08-17) already carries **four exact external identities as `4 × u24` in one +96-bit V3 facet** behind a `classid(4)`, refuse-don't-truncate, bake-time +crosswalk resolution. It strictly dominates the withdrawn type. Siblings: +`ogar_elk::ClassAddr` (`u32 + u32`, a pre-bake join key by its own doc), +`canonical_node::NodeGuid` + HHTL, the V3 OBO rail. + +Per the operator's rule — no concrete falsifier ⇒ **no new absolute-address +type.** S3.0 is closed as NOT-NEEDED, not filled because a plan had a slot. + +### The genuinely open addressing gap (different, not fixed here) + +`ClassId = u16` (`class_view.rs:54`) is near-exhausted for **relations** — +MedCare-rs `CLAUDE.md` #10: *"cannot address a relation — 11 prefixes, 8 of 280 +ids over the ceiling."* A classid-mint capacity question for OGAR/lance-graph; +to be raised with the operator in session, not patched from a consumer. + +### Where the work actually is + +| | ADDRESS | HYDRATED SoA | TRAVERSAL | +|---|---|---|---| +| Bible / Rosetta | yes | **NO** | partial / context | +| OSM | yes | yes | overlay / junction | +| MedCare ontology | yes | yes | **YES, Stage 1** | +| DisMech oracle | source | structured | causal oracle | + +The empty column is not ADDRESS. Next: **hydrate epistemic / causal nodes → +reason over them → think about the reasoning**, with the DisMech oracle +experiment as the gate (hide mechanism intermediates → hydrate the addressed +neighbourhood → let NARS/recipes recover candidates → compare against DisMech +truth). + +Full audit: `.claude/handovers/2026-08-20-s3-0-cold-start-recovery-audit.md`. +Stage-2/2.5/2.6 (#971) untouched; #970's CE64 layout untouched. + +--- ## 2026-08-20 — lance-graph #971 (MERGED, `2cbe62d`, head `d627f5c`) — Stage 2 carves + Stage 2.5 census + Stage 2.6a V3 invariance + CE64 ⇄ V3 losslessness Six commits, four stages. The per-stage Contract Inventory and results are in diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 5c29f4981..61dc53e5b 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -1,3 +1,33 @@ +## stage-3 — RE-SCOPED 2026-08-20: S3.0 closed as NOT-NEEDED; the ladder's empty column is HYDRATION, not ADDRESS + +PR #973 (S3.0 address, first attempt) closed unmerged. Its retraction PR then +attempted a corrected `CausalLiteral` and was **stopped before merge by the +operator** — four unearned claims, recorded in +`E-A-LOCAL-DERIVATION-CANNOT-OVERRULE-A-MEASURED-COUNTEREXAMPLE-1`. No new +address type is minted: `IdentityQuad` (4 × u24 in one V3 facet, ratified +2026-08-17) already carries exact four-component identity, and no falsifier +showed existing addressing insufficient. + +| D-id | Deliverable | Status | +|---|---|---| +| S3.0 | exact literal address | **CLOSED — NOT NEEDED** (use `IdentityQuad` / `ClassAddr` / V3 rail) | +| S3.0b | qualified causal regime | Queued | +| S3.1 | hydrate `CausalMeta` + `EpistemicMeta` over EXISTING addresses | **Next** | +| S3.1b | EntropyWork · BasinSet · Attention; `RowFocusMask × WideFieldMask` | Queued | +| S3.2 | V3 / CE64 leg from the hydrated node | Queued | +| S3.3 | `ResolvedPredicate`; unknown fails CLOSED; explicit composition | Queued | +| S3.4 | DisMech ORACLE experiment — hide intermediates, hydrate neighbourhood, NARS recovers, compare vs truth | **the gate** | +| S3.5 | NARS evidence mass: raw/source/HEEL/effective, deterministic W+/W− | Queued | +| S3.6 | JC measurement per predicate × cohort × horizon × instrument | Queued | +| S3.7 | `ReasoningEpisode`; measure the 17 confidence-mute kernels | Queued | +| S3.8 | potholes, first_possible vs first_derived, strict historical replay | Queued | +| S3.9 | Meta / Rubicon → OGAR-loco; Pearl qualification earned by receipts | Queued | + +**Open, and NOT an S3 deliverable:** `ClassId = u16` near-exhausted for +relations (MedCare-rs commitment #10) — an OGAR/lance-graph classid-mint +capacity question for the operator. + +--- ## preparation-arc plan wave — 2026-08-19 (operator: "integration plans for all open arcs") Five plans, each PROPOSED (no code — the reset charter's audit-first order diff --git a/.claude/handovers/2026-08-20-s3-0-cold-start-recovery-audit.md b/.claude/handovers/2026-08-20-s3-0-cold-start-recovery-audit.md new file mode 100644 index 000000000..07e83c875 --- /dev/null +++ b/.claude/handovers/2026-08-20-s3-0-cold-start-recovery-audit.md @@ -0,0 +1,192 @@ +# S3.0 Cold-Start Recovery Audit — 2026-08-20 + +**Outcome: no new address type. S3.0 as specified is WITHDRAWN.** +This PR retracts #973's overclaim, corrects a WordNet overclaim this session +itself introduced, and records why the S3.0 address slot needs no new type. + +## A. MERGED substrate truth + +- **PR #970** [MERGED, `781c3b9b`]: CE64 v2 layout final. Bits 59-60 = + `TrustTexture` (canonical) + ADDITIVE `CausalTopology` reading over the same + bits. Bits 61-63 = spare + ADDITIVE `ReasoningBand`, explicit + `with_reasoning_band()` only, never auto-derived. Deprecated v1 `temporal` + (52-63) is NOT valid v2 chronology; version-gate edges of unknown + provenance. Confirmed in `crates/causal-edge/src/layout.rs`. +- **PR #971** [MERGED]: Stage-2/2.5/2.6a frozen baseline. `InferenceType` is a + LOSSY compat projection of the 4-bit signed mantissa (8/16 states corrupted, + incl. `pack_v2` default `0 -> +1`) — carry the raw nibble. CE64<->V3 + bit-identical modulo dedup'd SPO + deprecated temporal. Capability != + reachability measured. +- **PR #973** [REJECTED, closed unmerged]: nothing on `main`. + +## B. MEASURED substrate truth + +1. **`identity_quad::IdentityQuad`** [MERGED, operator-RATIFIED 2026-08-17, + `ISS-IDENTITY-QUAD-WIDE-CARVING-HOME`]. **Four external identity spaces, + `4 × u24` contiguous, in ONE 96-bit V3 facet payload** via + `LegacyOutlier::WideTriple`, behind a `classid(4)`. Refuses rather than + truncates (`QuadError::OrdinalTooLarge`, `MAX_ORDINAL = 2^24 - 2`); + codebooks refuse rather than saturate (`CodebookError::TooLarge`). Its + stated purpose is resolving a crosswalk ONCE at bake time so a read is a + fixed-offset register read — no join, no crosswalk walk. +2. **`ogar_elk::ClassAddr`** [MERGED]: `classid: u32 + identity: u32` — 8 + bytes for ONE node, and its own doc is explicit that it is a pre-bake + **join key**, "not an ABI address, and deliberately not documented as one". +3. **Real OBO identities exceed `u16`** [MEASURED, MedCare-rs + `docs/ONTOLOGY_BAKE_STATE.md`:182]: *"real OBO ids run past `u16` + (MONDO:0700092 = 700,092) and one V3 field cannot hold that."* The + substrate ALREADY solved this: the V3 rail splits a 24-bit CURIE numeric as + `family:identity = (num >> 16, num & 0xFFFF)`, read via + `obo_store::row_addr`. Corpus scale: MONDO 32,095 · HP 19,836 · + UBERON 14,975 · PATO 1,887, and relations cross families. +4. **`ClassId = u16` is near-exhausted for RELATIONS** [MEASURED, MedCare-rs + `CLAUDE.md` commitment #10 + `RAIL_OFFENE_POSTEN.md`]: *"`ClassId = u16` + cannot address a relation — 11 prefixes, 8 of 280 ids over the ceiling."* + That is a real open gap — and it is a **classid-mint capacity** question + owned by OGAR/lance-graph, NOT something a new literal type fixes. +5. **WordNet #875/#876** [MERGED, MEASURED] — see §C2 for the corrected + reading. Also #876: a consumer can be structure-blind to an address's + hierarchy without that meaning the address lacks structure. +6. **TEKAMOLO #839/#844** [MERGED, live code]: one 16-byte content-blind + register, several simultaneous ClassView-selected readings + (`G3D4`/`G4D3`/`G6D2`/`24×i4`). Capability lands as a new READING. +7. **24×i4 anaphora #850** [MEASURED, 7,657 real German relative clauses, + 88.01% in-window]: a local representation exhausting cleanly marks a TYPE + boundary — switch reading, never widen the pointer. + +## C. REJECTED claims + +### C1. #973's `E-THE-LITERAL-CANNOT-LIVE-IN-THE-PATH-IT-ROOTS-1` — RETRACTED + +#973 measured correctly that a `domain·S·P·O` at `u16` each is exactly 16 +nibbles = `NiblePath::MAX_DEPTH`, zero slack. It then promoted that local fact +about ONE sequential depth-limited router path into an implied global claim +that exact identity cannot live in HHTL addressing and evidence must +"ref-escape" out of address space. The three counterexamples in §B5-B7 were +never consulted. The arithmetic survives; the inference does not. + +### C2. This session's OWN WordNet overclaim — RETRACTED + +The first draft of this recovery PR said #875 proved a *"full-width 4-ary HHTL +fold of real WordNet ancestry is an EXACT structural encoding"*. **That is +false, and #875's own numbers say so:** W5 reports *256/256 cells used, +occupancy min 29 / median 255 / max 1270* over **65,292 leaves**. That is +~255 leaves per cell — emphatically NOT injective, not an identity encoding. + +What #875 actually measured is a **deterministic, taxonomy-informed HHTL +locality / search prior**: shared-address-levels vs LCA-depth corr +0.494 (vs +−0.036 shuffled), out-of-cell band recall 0.763 vs 0.031 random = 24.71×, and +a 2.47-hop sub-nibble distinction the 16-ary router cannot address. A +discriminating prior that deliberately does NOT cover everything — #875 itself +dropped its cover guard as inert for exactly this reason. + +So the Bible/Rosetta + WordNet composition is: +`frozen verse identity × quasi-absolute taxonomy-informed semantic coordinate +× witness/qualia planes` — **not** evidence that 256 HHTL cells uniquely +identify a lexicon. Corrected in the PR body, EPIPHANIES, and this audit; +the module doc carrying it is deleted with the type. + +### C3. `CausalLiteral` was the wrong universal type — WITHDRAWN + +Two independent defects: + +- **Wrong name / wrong universality.** Its own test asserts + `CausalLiteral(7, 100, 43, 200)` as `TREATED_WITH` — demonstrating the + structure is generic, not causal. `ASSOCIATED_WITH` / `TREATED_WITH` / + `CAUSES` / `MEDIATES` / `PART_OF` are exact predicate identities over one + generic literal substrate. Causality is a **predicate family / + qualification**, never universal identity. If a primitive were needed it + would be `ExactLiteralAddr(D,S,P,O)`. +- **`4 × u16` is NOT absolute, and cannot be.** A `u16` subject cannot hold + MONDO:0700092 = 700,092 (§B3). The type would have silently mis-addressed + the single largest ontology in the MedCare bake. Calling it "absolute + identity" was a second global overclaim in the same PR that retracted one. + +## D. Salvageable + +- The nine tests' SHAPE (injectivity sweep, component-isolation matrix, + many-sources-one-literal, unbound sentinel) is good discipline and should be + reused by whatever type actually lands — but it belongs on + `IdentityQuad`/`ClassAddr`-based composition, not on a new 4×u16 plane. +- The retraction of C1 and the two process findings. +- `routing_prefix()` is REMOVED, not merely deferred — see §G. + +## E. The development ladder (operator-requested matrix) + +| | ADDRESS | HYDRATED SoA | TRAVERSAL | +|--------------------|---------|--------------|-----------| +| Bible / Rosetta | yes | **NO** | partial / context | +| OSM | yes | yes | overlay / junction | +| MedCare ontology | yes | yes | **YES, Stage 1** | +| DisMech oracle | source | structured | causal oracle | + +**The column that is empty is not ADDRESS.** Address is solved three times +over (`ClassAddr`, the V3 rail, `IdentityQuad`, `NodeGuid`/HHTL). The missing +work is: **hydrate epistemic / causal nodes → reason over them → think about +the reasoning.** + +### The question S3.0 had to answer, and its answer + +> **WHAT EXACT INFORMATION CANNOT BE EXPRESSED BY THE ADDRESSING THAT ALREADY +> EXISTS?** + +**Nothing that this session could demonstrate.** `IdentityQuad` (classid + 4 × +u24, ratified 2026-08-17) strictly dominates the withdrawn 4×u16 type on every +axis: wider (handles real OBO ids, which u16 does not), already V3-carving +conformant (rides a sanctioned reading rather than minting a parallel identity +plane), already refuse-don't-truncate, already bake-time-join-resolving. + +There is therefore **no concrete falsifier showing existing addressing is +insufficient**, and per the operator's rule no new absolute-address type is +minted. The S3.0 slot in the Stage-3 plan is closed as NOT-NEEDED rather than +filled because the plan had a slot. + +**The one genuinely open addressing gap found** is different and is NOT this: +`ClassId = u16` is near-exhausted for RELATIONS (§B4). That is a classid-mint +capacity question for OGAR/lance-graph, to be raised with the operator in +session — not something a new literal type addresses. + +## F. Next target (per operator §6) + +``` + EXISTING absolute address (ClassAddr / V3 rail / IdentityQuad / NodeGuid) + | + hydrate node + | + CausalMeta + EpistemicMeta + | + EntropyWork · BasinSet · Attention + | + RowFocusMask × WideFieldMask + | + V3 / CE64 -> NARS -> ReasoningEpisode -> Meta/Rubicon -> OGAR-loco +``` + +**DisMech as the oracle experiment:** hide known mechanism intermediates → +hydrate the addressed ontology neighbourhood → let NARS/recipes recover +candidates → compare against DisMech truth. Grounded: `dismech-rs` +`graph::build_causal_graph` is a real transcode of the upstream Python +resolver, falsified at **1,995 diseases / 33,458 edges** against the private +`medcare-dismech` measurement (33,328, within 0.4%), and 1,903 committed +`pathographs/MONDO_*.json` exist as ground truth. Not started this session. + +## G. `routing_prefix()` — REMOVED + +Concatenating D/S/P/O nibbles into a `NiblePath` yields deterministic +**lexicographic** prefixes. It was labelled an "HHTL locality / cohort +projection" — but no consumer exists and no measurement establishes that it +preserves HHTL *semantic* locality. `NiblePath` is built for the `subClassOf` +Abstammung tree, where a prefix is an ancestry claim; a lexicographic prefix +over concatenated ordinals carries no such guarantee. Calling it one was a +third unearned claim. Removed with the type; if it returns it needs a real +consumer and a measurement first. + +## H. Falsifiers that would reopen S3.0 + +1. A concrete case where `IdentityQuad` (4 × u24 + classid) provably cannot + express an exact proposition identity the substrate needs. +2. A measured need for a 4th component space beyond `IdentityQuad`'s four + slots — noting its own doc says a fifth identifier space is a **second + facet**, never a wider field. +3. Resolution of the `ClassId = u16` relation-capacity gap requiring a new + addressable relation identity (a classid-mint question first). diff --git a/crates/lance-graph-contract/src/dismech_evidence.rs b/crates/lance-graph-contract/src/dismech_evidence.rs new file mode 100644 index 000000000..c89f76ad2 --- /dev/null +++ b/crates/lance-graph-contract/src/dismech_evidence.rs @@ -0,0 +1,662 @@ +//! `dismech_evidence` — the compact evidence vocabulary measured on the real +//! DisMech corpus, plus the typed citation sidecar. +//! +//! **Source-side only.** This module encodes what the DisMech YAML *says*. It +//! deliberately does NOT reference `CausalEdge64`: the durable causal overlay +//! must not become a pile of hot reasoning registers (operator ruling). The +//! mapping [`DismechTopology`] → CE64 bits 59..60 happens at HYDRATION, in the +//! consumer, and is a 1:1 read of [`DismechTopology::to_bits_2`]. +//! +//! # Measured, not designed (upstream `monarch-initiative/dismech`, 2,100 files) +//! +//! Every cardinality below was counted on the real corpus before this module +//! existed; each enum is exhaustively round-tripped in the tests. +//! +//! | field | states | bits | occurrences | +//! |---|---|---|---| +//! | `causal_link_type` | 4 | 2 | **17,998** | +//! | `supports` | 4 | 2 | ~89,800 | +//! | `evidence_source` | 5 | 3 | ~79,200 | +//! | `modifier` | 7 | 3 | 9,926 | +//! | `frequency` | 19 | 5 | 11,767 | +//! +//! `causal_link_type`'s four values are *exactly* the four `CausalTopology` +//! states, so the mapping is **source-authoritative** — never inferred from +//! confidence, edge count, or predicate name. +//! +//! ## Why every parse FAILS CLOSED +//! +//! An unrecognised source token returns `None`. It must never fold into a +//! neighbouring state and must never default to `Unknown`: `UNKNOWN` is a value +//! the corpus actually asserts (408 topology rows), so silently minting it from +//! a parse failure would forge an assertion the source never made. +//! +//! # The citation sidecar +//! +//! Measured: **131,904** reference-title occurrences over **31,361** distinct +//! titles (4.21× reuse); 12.12 MB inline → 3.05 MB deduped. But titles are the +//! WRONG key — the corpus is LLM-generated, so wording drifts between +//! regenerations while the citation identity does not. The corpus already +//! carries stable identifiers on ~104,700 occurrences: PMID (dominant), DOI, +//! ORPHA, CGGV, ClinicalTrials, URL. +//! +//! So a [`CitationKey`] is `(namespace, id)`, and the title is COLD content +//! reached by [`ContentId`]. Where no stable identifier exists, +//! [`CitationKey::ContentAddressed`] says so **explicitly** — a fake +//! bibliographic identity is never synthesised from a title. + +use crate::content_store::ContentId; + +/// Source `causal_link_type` — the four states, 2 bits. +/// +/// Maps 1:1 onto the CE64 `CausalTopology` ordinal at hydration; the mapping is +/// source-authoritative and is never inferred. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)] +#[repr(u8)] +pub enum DismechTopology { + /// `DIRECT` — measured 9,073. + Direct = 0, + /// `INDIRECT_KNOWN_INTERMEDIATES` — measured 3,978. The hidden-mediator + /// oracle population: the source names the intermediates, so they can be + /// hidden and recovery measured. + IndirectKnownIntermediates = 1, + /// `INDIRECT_UNKNOWN_INTERMEDIATES` — measured 4,539. The genuine + /// known-unknown population and the epistemic-restraint control: the source + /// itself does not know the mediator, so a reasoner that "recovers" one is + /// hallucinating closure. + IndirectUnknownIntermediates = 2, + /// `UNKNOWN` — measured 408. An asserted value, never a parse fallback. + Unknown = 3, +} + +impl DismechTopology { + /// All four, ordinal order. + pub const ALL: [Self; 4] = [ + Self::Direct, + Self::IndirectKnownIntermediates, + Self::IndirectUnknownIntermediates, + Self::Unknown, + ]; + + /// Parse the source token. **Fails closed** — no folding, no default. + #[must_use] + pub fn from_source(s: &str) -> Option { + Some(match s { + "DIRECT" => Self::Direct, + "INDIRECT_KNOWN_INTERMEDIATES" => Self::IndirectKnownIntermediates, + "INDIRECT_UNKNOWN_INTERMEDIATES" => Self::IndirectUnknownIntermediates, + "UNKNOWN" => Self::Unknown, + _ => return None, + }) + } + + /// The exact source token — round-trips [`from_source`](Self::from_source). + #[must_use] + pub const fn as_source(self) -> &'static str { + match self { + Self::Direct => "DIRECT", + Self::IndirectKnownIntermediates => "INDIRECT_KNOWN_INTERMEDIATES", + Self::IndirectUnknownIntermediates => "INDIRECT_UNKNOWN_INTERMEDIATES", + Self::Unknown => "UNKNOWN", + } + } + + /// The 2-bit ordinal. Consumers map this onto CE64 bits 59..60; this module + /// never does, so the source bake stays free of hot registers. + #[must_use] + pub const fn to_bits_2(self) -> u8 { + self as u8 + } + + /// Inverse of [`to_bits_2`](Self::to_bits_2). Fails closed above 3. + #[must_use] + pub const fn from_bits_2(b: u8) -> Option { + Some(match b { + 0 => Self::Direct, + 1 => Self::IndirectKnownIntermediates, + 2 => Self::IndirectUnknownIntermediates, + 3 => Self::Unknown, + _ => return None, + }) + } + + /// Does the SOURCE claim to know the intermediates? + /// + /// True only for [`IndirectKnownIntermediates`](Self::IndirectKnownIntermediates). + /// This is what separates the oracle population from the restraint control: + /// for the other three a recovered mediator is not a success, it is an + /// unsupported claim. + #[must_use] + pub const fn source_knows_intermediates(self) -> bool { + matches!(self, Self::IndirectKnownIntermediates) + } + + /// Is the mediator slot legitimately unresolved in the SOURCE? + /// + /// **[`Unknown`](Self::Unknown) is deliberately NOT included**, and the + /// distinction is epistemic rather than cosmetic: + /// + /// ```text + /// IndirectUnknownIntermediates A -> ? -> B + /// the path is known indirect, the mediator ROLE is established, + /// only its IDENTITY is missing => SeekMediator is warranted + /// + /// Unknown A ? B + /// direct-vs-indirect is itself undecided, so the mediator role is + /// not established at all => SeekMediator would MINT a schema + /// slot the source never asserted + /// ``` + /// + /// Conflating them lets `Unknown` dispatch a mediator search for a hole + /// nobody claimed exists. Ask [`topology_unresolved`](Self::topology_unresolved) + /// for that case instead. + /// + /// **This predicate answers what the SOURCE asserts, never whether a + /// mediator is actually present**, and on real data those differ sharply. + /// + /// Measured on `monarch-initiative/dismech` (2,100 disorder files), + /// **scoped to `pathophysiology[].downstream[]`** — the edge list that + /// actually carries `intermediate_mechanisms`. This is a SUBSET of the + /// corpus-wide `causal_link_type` census in the module header (which counts + /// the field wherever it occurs), so the totals differ by scope, not by + /// disagreement: 3,844 here against 3,978 corpus-wide. + /// + /// | declared | with ≥1 intermediate | with ZERO | + /// |---|---|---| + /// | `INDIRECT_KNOWN_INTERMEDIATES` (3,844) | 2,497 | **1,347 = 35.0%** | + /// | `INDIRECT_UNKNOWN_INTERMEDIATES` (4,325) | 92 (contradictory) | 4,233 | + /// + /// So a third of the edges that CLAIM known intermediates list none, and a + /// few that claim none list some. A consumer building the hidden-mediator + /// oracle must filter on the ACTUAL intermediate list, never on the topology + /// ordinal alone: the usable population is **2,497 distinct + /// `(disease, source, target, mediators)` tuples across 539 diseases** + /// (deduped — the raw and distinct counts coincide, so no many-to-many + /// mention inflation), not 3,844. + #[must_use] + pub const fn mediator_unresolved(self) -> bool { + matches!(self, Self::IndirectUnknownIntermediates) + } + + /// Is the TOPOLOGY ITSELF unresolved — i.e. the source has not decided + /// direct vs indirect, so no mediator role is established? + /// + /// The companion to [`mediator_unresolved`](Self::mediator_unresolved): + /// together they partition the two genuinely-unresolved states without + /// collapsing them. Measured 403 `UNKNOWN` against 4,325 + /// `INDIRECT_UNKNOWN_INTERMEDIATES` — a ~10.7x population difference that a + /// merged predicate would hide. + #[must_use] + pub const fn topology_unresolved(self) -> bool { + matches!(self, Self::Unknown) + } +} + +/// Source `supports` — 4 states, 2 bits. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[repr(u8)] +pub enum Supports { + /// `SUPPORT`. + Support = 0, + /// `PARTIAL`. + Partial = 1, + /// `REFUTE`. + Refute = 2, + /// `NO_EVIDENCE`. + NoEvidence = 3, +} + +impl Supports { + /// All four. + pub const ALL: [Self; 4] = [Self::Support, Self::Partial, Self::Refute, Self::NoEvidence]; + + /// Parse; fails closed. + #[must_use] + pub fn from_source(s: &str) -> Option { + Some(match s { + "SUPPORT" => Self::Support, + "PARTIAL" => Self::Partial, + "REFUTE" => Self::Refute, + "NO_EVIDENCE" => Self::NoEvidence, + _ => return None, + }) + } + + /// The exact source token. + #[must_use] + pub const fn as_source(self) -> &'static str { + match self { + Self::Support => "SUPPORT", + Self::Partial => "PARTIAL", + Self::Refute => "REFUTE", + Self::NoEvidence => "NO_EVIDENCE", + } + } + + /// 2-bit ordinal. + #[must_use] + pub const fn to_bits_2(self) -> u8 { + self as u8 + } +} + +/// Source `evidence_source` — 5 states, 3 bits. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[repr(u8)] +pub enum EvidenceSource { + /// `HUMAN_CLINICAL`. + HumanClinical = 0, + /// `MODEL_ORGANISM`. + ModelOrganism = 1, + /// `IN_VITRO`. + InVitro = 2, + /// `COMPUTATIONAL`. + Computational = 3, + /// `OTHER`. + Other = 4, +} + +impl EvidenceSource { + /// All five. + pub const ALL: [Self; 5] = [ + Self::HumanClinical, + Self::ModelOrganism, + Self::InVitro, + Self::Computational, + Self::Other, + ]; + + /// Parse; fails closed. `OTHER` is an asserted value, never a fallback. + #[must_use] + pub fn from_source(s: &str) -> Option { + Some(match s { + "HUMAN_CLINICAL" => Self::HumanClinical, + "MODEL_ORGANISM" => Self::ModelOrganism, + "IN_VITRO" => Self::InVitro, + "COMPUTATIONAL" => Self::Computational, + "OTHER" => Self::Other, + _ => return None, + }) + } + + /// The exact source token. + #[must_use] + pub const fn as_source(self) -> &'static str { + match self { + Self::HumanClinical => "HUMAN_CLINICAL", + Self::ModelOrganism => "MODEL_ORGANISM", + Self::InVitro => "IN_VITRO", + Self::Computational => "COMPUTATIONAL", + Self::Other => "OTHER", + } + } + + /// 3-bit ordinal. + #[must_use] + pub const fn to_bits_3(self) -> u8 { + self as u8 + } +} + +/// Where a citation's identity comes from. +/// +/// The namespace is part of the KEY, not a hint — `PMID:123` and `ORPHA:123` +/// are different citations. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)] +#[repr(u8)] +pub enum CitationNamespace { + /// PubMed id — dominant in the corpus. + Pmid = 0, + /// DOI. + Doi = 1, + /// Orphanet. + Orpha = 2, + /// ClinicalTrials.gov (`NCT…`). + ClinicalTrials = 3, + /// ClinGen/CGGV. + Cggv = 4, + /// A bare URL. + Url = 5, +} + +impl CitationNamespace { + /// All six. + pub const ALL: [Self; 6] = [ + Self::Pmid, + Self::Doi, + Self::Orpha, + Self::ClinicalTrials, + Self::Cggv, + Self::Url, + ]; + + /// Parse a CURIE prefix (case-insensitive); fails closed. + #[must_use] + pub fn from_prefix(p: &str) -> Option { + Some(match p.to_ascii_uppercase().as_str() { + "PMID" => Self::Pmid, + "DOI" => Self::Doi, + "ORPHA" => Self::Orpha, + "NCT" | "CLINICALTRIALS" => Self::ClinicalTrials, + "CGGV" => Self::Cggv, + "URL" | "HTTP" | "HTTPS" => Self::Url, + _ => return None, + }) + } + + /// The canonical prefix. + #[must_use] + pub const fn prefix(self) -> &'static str { + match self { + Self::Pmid => "PMID", + Self::Doi => "DOI", + Self::Orpha => "ORPHA", + Self::ClinicalTrials => "NCT", + Self::Cggv => "CGGV", + Self::Url => "URL", + } + } +} + +/// The stable identity of one cited source. +/// +/// **Identity never derives from the title.** The corpus is LLM-generated, so +/// title wording drifts between regenerations while the citation does not — +/// pinned by `reference_identity_survives_a_title_rewording`. +#[derive(Clone, Debug, PartialEq, Eq, Hash)] +pub enum CitationKey { + /// A real, stable external identifier. + Identified { + /// Which identifier space. + namespace: CitationNamespace, + /// The bare id, prefix stripped (e.g. `"25252"`). + id: String, + }, + /// No stable identifier exists — the content address of the citation text, + /// **explicitly marked as such**. Never a synthesised bibliographic id. + ContentAddressed(ContentId), +} + +impl CitationKey { + /// Parse a source `reference` value such as `"PMID:25252"`. + /// + /// Falls back to [`ContentAddressed`](Self::ContentAddressed) when the value + /// carries no recognised namespace — deliberately explicit, so a consumer + /// can always tell a real citation from a hashed string. + /// + /// Returns `None` for a blank or whitespace-only `raw`. **Absence is made + /// structurally unrepresentable**, because the alternative is worse than it + /// looks: `fnv1a(b"")` is the nonzero FNV offset basis + /// `0xcbf2_9ce4_8422_2325`, so hashing `""` would mint a perfectly + /// well-formed [`ContentAddressed`](Self::ContentAddressed) key meaning + /// "no citation supplied" — indistinguishable on replay from a real + /// content-addressed citation, and colliding every uncited row onto one + /// shared identity. `ContentId(0)` is the reserved sentinel + /// ([`ContentId::is_sentinel`]), and `of_str("")` does not produce it. + #[must_use] + pub fn parse(raw: &str) -> Option { + let t = raw.trim(); + if t.is_empty() { + return None; + } + if let Some((p, rest)) = t.split_once(':') { + if let Some(ns) = CitationNamespace::from_prefix(p) { + let id = if matches!(ns, CitationNamespace::Url) { + t.to_string() + } else { + rest.trim().to_string() + }; + if !id.is_empty() { + return Some(Self::Identified { namespace: ns, id }); + } + } + } + Some(Self::ContentAddressed(ContentId::of_str(t))) + } + + /// Does this key rest on a real external identifier? + #[must_use] + pub const fn is_identified(&self) -> bool { + matches!(self, Self::Identified { .. }) + } +} + +/// One bibliography row: a stable key, with the title held as COLD content. +/// +/// The title is never stored inline — 131,904 occurrences over 31,361 distinct +/// titles measured, so inlining costs 12.12 MB to say 3.05 MB. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct BibliographyRecord { + /// Stable identity. + pub key: CitationKey, + /// CAM handle for the title text, or the reserved sentinel `ContentId(0)` + /// (see [`ContentId::is_sentinel`]) when no title was supplied. + /// + /// The sentinel must be written explicitly: `of_str("")` hashes the empty + /// byte string to the nonzero FNV offset basis, so an untitled record would + /// otherwise carry a real-looking content address that no store can resolve. + pub title: ContentId, +} + +impl BibliographyRecord { + /// Build from a raw `reference` and its (possibly drifting) title. + /// + /// `None` when `reference` is blank — a bibliography row with no citation + /// is not a row (see [`CitationKey::parse`]). A blank `title` is legal and + /// lands on the `ContentId(0)` sentinel rather than on `fnv1a(b"")`. + #[must_use] + pub fn new(reference: &str, title: &str) -> Option { + Some(Self { + key: CitationKey::parse(reference)?, + title: if title.trim().is_empty() { + ContentId(0) + } else { + ContentId::of_str(title) + }, + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// Every source token round-trips, and the ordinal is stable. + #[test] + fn topology_round_trips_every_source_value() { + for t in DismechTopology::ALL { + assert_eq!(DismechTopology::from_source(t.as_source()), Some(t)); + assert_eq!(DismechTopology::from_bits_2(t.to_bits_2()), Some(t)); + } + // anti-vacuity: the four are genuinely distinct ordinals + let mut bits: Vec = DismechTopology::ALL.iter().map(|t| t.to_bits_2()).collect(); + bits.sort_unstable(); + bits.dedup(); + assert_eq!(bits.len(), 4, "topology ordinals collided"); + } + + /// FAIL CLOSED — and specifically, never mint `Unknown`, which the corpus + /// asserts 408 times and which would forge an assertion. + #[test] + fn unrecognised_topology_fails_closed_and_never_becomes_unknown() { + for bad in [ + "", + "direct", + "INDIRECT", + "INDIRECT_KNOWN", + "UNKNOWN_INTERMEDIATES", + "unknown", + ] { + assert_eq!( + DismechTopology::from_source(bad), + None, + "{bad:?} must not parse" + ); + } + assert_eq!(DismechTopology::from_bits_2(4), None); + // the silence half: the real token still parses + assert_eq!( + DismechTopology::from_source("UNKNOWN"), + Some(DismechTopology::Unknown) + ); + } + + /// The two experimental populations are distinguishable, which is what makes + /// the hidden-mediator test and its restraint control separable. + #[test] + fn only_indirect_known_claims_the_source_knows_intermediates() { + assert!(DismechTopology::IndirectKnownIntermediates.source_knows_intermediates()); + for t in [ + DismechTopology::Direct, + DismechTopology::IndirectUnknownIntermediates, + DismechTopology::Unknown, + ] { + assert!(!t.source_knows_intermediates(), "{t:?} must not claim it"); + } + assert!(DismechTopology::IndirectUnknownIntermediates.mediator_unresolved()); + assert!(!DismechTopology::Direct.mediator_unresolved()); + } + + #[test] + fn supports_and_evidence_source_round_trip_every_value() { + for s in Supports::ALL { + assert_eq!(Supports::from_source(s.as_source()), Some(s)); + assert!(s.to_bits_2() < 4); + } + for e in EvidenceSource::ALL { + assert_eq!(EvidenceSource::from_source(e.as_source()), Some(e)); + assert!(e.to_bits_3() < 8); + } + assert_eq!(Supports::from_source("MAYBE"), None); + assert_eq!(EvidenceSource::from_source("HUMAN"), None); + } + + /// THE LLM-DRIFT FALSIFIER: identity is the citation, not the wording. + #[test] + fn reference_identity_survives_a_title_rewording() { + let a = BibliographyRecord::new( + "PMID:25252", + "Insulin resistance in type 2 diabetes mellitus", + ) + .expect("a real citation"); + let b = BibliographyRecord::new( + "PMID:25252", + "Insulin Resistance in Type 2 Diabetes Mellitus: A Review", + ) + .expect("a real citation"); + assert_eq!(a.key, b.key, "citation identity must survive a re-wording"); + assert_ne!(a.title, b.title, "the titles genuinely differ"); + // …and a different citation is a different key even with the same title + let c = BibliographyRecord::new( + "PMID:99999", + "Insulin resistance in type 2 diabetes mellitus", + ) + .expect("a real citation"); + assert_ne!(a.key, c.key); + assert_eq!(a.title, c.title); + } + + /// The namespace is part of the key — same digits, different source. + #[test] + fn namespace_is_part_of_the_identity() { + assert_ne!( + CitationKey::parse("PMID:123").unwrap(), + CitationKey::parse("ORPHA:123").unwrap() + ); + for ns in CitationNamespace::ALL { + assert_eq!(CitationNamespace::from_prefix(ns.prefix()), Some(ns)); + } + assert_eq!(CitationNamespace::from_prefix("PUBMED"), None); + } + + /// An unidentified citation says so, rather than pretending to a + /// bibliographic id synthesised from its title. + #[test] + fn unidentified_citations_are_explicitly_content_addressed() { + let k = CitationKey::parse("Smith et al, personal communication") + .expect("non-blank input parses"); + assert!( + !k.is_identified(), + "must not claim a bibliographic identity" + ); + assert!(matches!(k, CitationKey::ContentAddressed(_))); + // deterministic + assert_eq!( + k, + CitationKey::parse("Smith et al, personal communication").unwrap() + ); + assert!(CitationKey::parse("PMID:25252").unwrap().is_identified()); + } + /// P1 FALSIFIER — `Unknown` must NOT dispatch a mediator search. + /// + /// `IndirectUnknownIntermediates` establishes the mediator ROLE and leaves + /// its identity open; `Unknown` leaves direct-vs-indirect itself undecided, + /// so treating it as an unresolved mediator MINTS a schema slot the source + /// never asserted. Two-sided, and the silence half is the point. + #[test] + fn unknown_topology_is_not_an_unresolved_mediator() { + assert!(DismechTopology::IndirectUnknownIntermediates.mediator_unresolved()); + assert!( + !DismechTopology::Unknown.mediator_unresolved(), + "Unknown must not claim a mediator hole the source never established" + ); + // …and the companion predicate covers it instead, exclusively. + assert!(DismechTopology::Unknown.topology_unresolved()); + assert!(!DismechTopology::IndirectUnknownIntermediates.topology_unresolved()); + // the two resolved states answer NEITHER + for t in [ + DismechTopology::Direct, + DismechTopology::IndirectKnownIntermediates, + ] { + assert!(!t.mediator_unresolved(), "{t:?} is not mediator-unresolved"); + assert!(!t.topology_unresolved(), "{t:?} is not topology-unresolved"); + } + // anti-vacuity: the two predicates are disjoint and neither is constant + let any_med = DismechTopology::ALL + .iter() + .filter(|t| t.mediator_unresolved()) + .count(); + let any_top = DismechTopology::ALL + .iter() + .filter(|t| t.topology_unresolved()) + .count(); + assert_eq!( + (any_med, any_top), + (1, 1), + "each predicate names exactly one state" + ); + } + + /// P1 FALSIFIER — absence must not become a well-formed content address. + /// + /// `fnv1a(b"")` is the nonzero FNV offset basis, so hashing a blank would + /// mint a real-looking citation meaning "none supplied" and collide every + /// uncited row onto it. + #[test] + fn a_blank_citation_is_absent_not_content_addressed() { + for blank in ["", " ", "\t\n "] { + assert_eq!( + CitationKey::parse(blank), + None, + "blank {blank:?} must not parse to a citation" + ); + assert_eq!(BibliographyRecord::new(blank, "a title"), None); + } + // the trap this guards, stated as a measurement rather than a memory: + assert_ne!( + ContentId::of_str("").0, + 0, + "of_str(\"\") is the FNV offset basis, NOT the sentinel" + ); + // a blank TITLE is legal, and lands on the sentinel rather than the basis + let r = BibliographyRecord::new("PMID:25252", " ").expect("real citation"); + assert!( + r.title.is_sentinel(), + "an untitled record uses ContentId(0)" + ); + assert_ne!(r.title, ContentId::of_str(""), "must not be the empty hash"); + // anti-vacuity: a real title is NOT the sentinel + let t = BibliographyRecord::new("PMID:25252", "A real title").expect("real citation"); + assert!(!t.title.is_sentinel()); + } +} diff --git a/crates/lance-graph-contract/src/lib.rs b/crates/lance-graph-contract/src/lib.rs index 02d2a4451..c1c88b5ce 100644 --- a/crates/lance-graph-contract/src/lib.rs +++ b/crates/lance-graph-contract/src/lib.rs @@ -77,6 +77,7 @@ pub mod cycle_accumulator; /// `SquishedDawg` (dict/dawg binary loader) — Tesseract dictionary word-graph /// content store. See module docs for the `dawg.{h,cpp}` byte-parity scope. pub mod dawg; +pub mod dismech_evidence; pub mod dispatch_guard; pub mod dispatch_mode; pub mod distance;