From a11423c224e092062820d6edec06747e80831ccb Mon Sep 17 00:00:00 2001 From: John Lambert Date: Thu, 27 Aug 2026 11:43:58 -0400 Subject: [PATCH 1/5] docs: HermitCrab optimization ledger -- 22 attempts, tried/closed/open Single consolidated report from a measurement effort across 33 conformance grammars plus Sena, Indonesian and Amharic. One row per optimization attempted, with what was expected, why it failed, and the number that settled it. Net speed delivered: none. The fastest thing tried was 4% slower. What the work produced is the measurement, a closed family of optimizations, a located target, and this ledger. Headline findings: - Synthesis is ~5% of Sena and 0.3% of Amharic wall time. Four prior optimization rounds and the originating proposal were aimed at the wrong phase. - The redundancy in synthesis is apparent, not real: three independent measurements (9,774x, 6,476x, 3.22x/8.10x) all collapsed once the key carried the trail, the last to hits=0. That closes packed forests, fold sharing and synthesis-input dedupe as a family. - Only ~16% of the Amharic analysis cascade is linguistic computation; clone plus unattributed orchestration is 84%. Docs-only by design. All instrumentation stays on the four named branches: the probes gate on volatile bool reads inside Matcher.cs, a hot inner loop in SIL.Machine, which is fine for a measurement run and not something to carry in product code. Rebuilding a probe from a branch is cheap. Records two durable code constraints, a calibration record of predictions vs outcomes, and eight method rules -- including that a bucket reporting exactly 0.00 is broken until proven otherwise, and that an explicit remainder column would have caught four of the five retractions at the point of measurement. Co-Authored-By: Claude Opus 5 --- docs/hermitcrab-optimization-ledger.md | 200 +++++++++++++++++++++++++ 1 file changed, 200 insertions(+) create mode 100644 docs/hermitcrab-optimization-ledger.md diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md new file mode 100644 index 00000000..ca1ccac9 --- /dev/null +++ b/docs/hermitcrab-optimization-ledger.md @@ -0,0 +1,200 @@ +# HermitCrab optimization ledger — tried, closed, do not retry + +**Purpose: stop this work being redone.** One row per optimization attempted, what was expected, +what actually happened, and the number that settled it. If you are about to try something on this +list, read its row first — several of these look irresistible on paper and three of them have +already been independently rediscovered and re-retracted. + +Deliberately in-repo and durable, not evicted into a PR accordion. A closed avenue is only closed +if the next person can find out cheaply that it is closed. + +**This is the only report.** The working docs it was distilled from +(`hermitcrab-packed-forest-research.md`, `hermitcrab-forest-memo-plan.md`, +`hermitcrab-forest-memo-ceiling.md`, `hermitcrab-synthesis-fold-probes.md`) are chronological +process records containing retracted intermediate claims, and are deliberately **not** merged. +They stay on the branches below, together with all instrumentation, as reproducible evidence: + +| branch | holds | +| --- | --- | +| `feature/forest-memo` | key-narrowing measurement, `RuleLengthClassifier` + 12 tests | +| `feature/synthesis-fold-probes` | the eight-bucket wall-time split, die-point histogram, sharing census, harness | +| `feature/synthesis-fold-sharing` | `SynthesisStateKey` + fold memo, parity-clean, `hits = 0` | +| `feature/per-node-cost` | `NodeCostProbe` (SIL.Machine layer) + the per-node decomposition | + +**No instrumentation is merged, by design.** The probes gate on `volatile bool` reads inside +`Matcher.cs` — a hot inner loop in `SIL.Machine` used by every consumer — which is an acceptable +cost for a measurement run and not one to carry in product code indefinitely. Rebuilding a probe +from a branch is cheap; maintaining research scaffolding in two product assemblies is not. + +## Net speed delivered: none + +No optimization in the "closed" section shipped, and none of them made anything faster. The +fastest thing tried was 4% **slower**. The pre-existing wins in this codebase (rows 1–3) predate +this work. What this effort produced is the measurement, the closed family, the located target, +and this ledger. + +--- + +## Shipped and working + +| # | Optimization | Expected | Outcome | Number | +| --- | --- | --- | --- | --- | +| 1 | Memoize analysis morphological-rule cascade | Same state reached by many rule orders; cache it | Works, but the cascade was never the cost | 2,555 expansions vs a 2,546-state floor; 1.4 s of a 30.5 s word | +| 2 | Memoize affix-template battery (same key) | Battery was 93% of wall time | **The big win.** Reduced how *often* the battery runs | 38,840 runs -> 2,581; word 30.5 s -> **6.1 s** | +| 3 | Shape sharing at clone (`CloneShareFrozenShape`) | Eager deep-copy of Shape dominates allocation | Works, allocation only | **-4.5 to -7.6% bytes**, no wall change | + +## Closed — measured, does not pay + +| # | Optimization | Expected | Why it failed | Number | +| --- | --- | --- | --- | --- | +| 4 | **Narrow `AnalysisStateKey`** (drop strictly-shrinking rules' un-application counts) | `MaxApplicationCount` defaults to 1, so the count multiset is "which subset of rules was used" — 2^27 states. Dropping 19 of 27 rules gives 2^8 | Shape + syntactic FS + realizational FS *already* discriminate. The count component was implied by its neighbours. A worst-case bound collapsing is not the realised state count collapsing | Realised state collapse **Sena 1.12x** vs a 1.3 gate; 6 of 7 heavy words exactly **1.00**; `atawirambo` 2,556 -> 2,556 | +| 5 | **Fold-step sharing memo** (share deterministic synthesis fold steps by computed value) | Probe measured 3.22x and 8.10x shareable steps; ~50% wall ceiling on two reliable fixtures | A sound key must carry the **ordered remaining trail**. Two candidates then have to agree on their entire future to share a step, which almost never happens | **hits = 0** on the best fixture; **0.96x** (4.5% *slower* — pure key-construction cost); 1.06x on the second, inside 21.9% noise | +| 6 | **Order-insensitive synthesis-input dedupe** | 9,774x duplicate synthesis inputs | HermitCrab's morphological rules are **not order-invariant**: same rule multiset, different order, different output. Merging loses parses | 2 violations Sena, 12 independently Indonesian (e.g. `{meN, -Cont}`). Sound version: **15–40%** | +| 7 | **Trail-position indexing of synthesis rules** | 11,445,538 of 11,445,538 rejections are "this rule is not the pending trail rule" — 100% of the histogram | Each rejection is one array index plus one reference compare. Count is not cost | 11.4M x 29 ns = 0.33 s of 143 s = **0.2%** | +| 8 | Synthesis-side length bound (Gate A) | Reject candidates that cannot reach the surface length | At the comparison point the candidate is still the bare root; its affix trail applies later inside `_synthesisRule.Apply`. Rejected valid parses | Unit suite 64 -> 34 passing. Reverted | +| 9 | Analysis length ceiling (Gate B) | Same idea, analysis side | Sound and correct. No available corpus contains the pathology it prunes | Byte-identical output, **no measurable speedup** (~4–7% slower, within noise) | +| 10 | Lexical reachability gate (Phase 5) | Prune candidates from which no lexicon root is reachable | Both reference corpora have real compounding in their deepest stratum, which disqualifies the gate everywhere | Proven **no-op** on both corpora | +| 11 | Tandem lexical intersection (T2) | Kill doomed branches early via the lexicon | The oracle only sees lexical unreachability. The expensive words fail on checks that run *after* lexical lookup succeeds | Pooled **23.5%** dead steps vs a 30% gate; one failure word at exactly **0.0%** dead | +| 12 | Pool small short-lived collections | Fewer allocations, faster | `HashSet/Dictionary.Clear()` is O(capacity); Gen0 already beats pooling at this size | -15–17% bytes but **+8.6% wall**, +12% on the parallel battery. Reverted | + +## Closed before building — motivation refuted by measurement + +| # | Optimization | Expected | Why it died | +| --- | --- | --- | --- | +| 13 | Surface-length pruning inside the synthesis fold (P2) | "The only lever on Amharic's ~160 ms per synthesis run" | Amharic's forward synthesis is **14 ms**. The 160 ms was arithmetic on an unmeasured denominator — dividing 30 s by 186 synthesis inputs and assuming the time was there | +| 14 | Nogood lattice subsumption (P4) | Nogood hits dominate real hits 434,628 to 25,102; generalise "empty" over the feature lattice | Shape + features already discriminate almost perfectly (see #4), so subsumption buckets hold one member and nothing generalises | +| 15 | Move constraints into the analysis key (Maxwell & Kaplan's category-splitting) | Their biggest measured win: make the chart prune what the constraint solver otherwise would | Everything synthesis rejects on is root- or realization-dependent, and the root is unknown until `LexicalLookup`. Also a Sena analysis state costs ~0.73 ms while a synthesis input costs at most ~0.12 ms — **states are dearer than synthesis inputs** | + +## Open + +| # | Optimization | Status | +| --- | --- | --- | +| 21 | **Memo-key caching on the frozen `Word`** | **CLOSED.** Keys are built ~28x per distinct state, which looked like a cheap win. Measured cost of all `AnalysisStateKey` construction and hashing on Amharic: **3 ms of 81,707 ms = 0.0%.** Same error shape as #7: high count, no cost | +| 22 | **Analysis-side rule prefilter** (index rules by segment/feature signature so a rule that cannot match is never handed to the matcher) | **Bounded, not built.** 92.3% of allomorph pattern matches fail (23,240 of 25,178) and 80.5% of rule attempts reach matching — so the prefilter target is real. But matcher traversal is only **12.5%** of the Amharic cascade, capping this at ~11.5% there. Worth more on the template battery, where matching is **39.3%** | +| 16 | **`ExpandAlternatives` dedupe** | Live. 20.3% of Sena wall time, never instrumented before this work. 926 alternatives -> 3 distinct on one fixture, **100% same-analysis-word** so interceptable pre-expansion. Sound ceiling **3–8% Sena**. Needs a trail-complete key | +| 17 | **Per-node cost in the analysis cascade** | Live and unexplored — see below | +| 18 | Corpus-scope memoization (P3) | Not run. `AnalysisScope` dies per word but the key is word-independent by construction. Amharic's corpus run takes 4.3 hours | +| 19 | Contexted constraints / abstract feature-only replay | Not built. Gated on a die-point histogram that says candidates die on shape-free checks | +| 20 | Generation (`GenerateWords`) | **Unmeasured.** Pure synthesis, no analysis phase, so share ~100% — the fold-step ratios would apply at face value. Every verdict above is scoped to *parsing* | + +--- + +## The lesson that generalises + +Three independent measurements, three boundaries, same collapse: + +| measurement | key used | apparent | sound | +| --- | --- | --- | --- | +| synthesis-input dedupe (#6) | order-insensitive | 9,774x | 15–40% | +| fold-entry census | trail position only | 6,476x | not established | +| fold-step sharing (#5) | trail position only | 3.22x / 8.10x | **hits = 0** | + +**The redundancy in HermitCrab's synthesis is apparent, not real. The trail is what makes each +step distinct, and every measurement showing large shareable work is measuring a key that omits +it.** The thing that makes sharing visible is the thing that makes sharing wrong. + +That closes a *family*: packed parse forests, fold-step sharing, and synthesis-input dedupe all +require distinct derivations to converge on a genuinely identical state. In this engine they do +not converge. It is the same fact as the rules being non-order-invariant, seen from the other side. + +## Where the time actually is + +Measured with eight exclusive buckets summing to wall (`unaccounted` 0.1% on Amharic, 1.5% on Sena): + +| grammar | breakdown | +| --- | --- | +| **Sena** heavy words | analysis template battery **51.4%** (still, after the 5x memo), `ExpandAlternatives` **20.3%**, analysis cascade 18.4%, **all synthesis ~5%** | +| **Amharic** | analysis cascade **~95%** of analysis, which is 99.5% of wall; **all synthesis 0.3%** | + +**Every optimization in this ledger that worked reduced how many nodes are visited. None touched +what a node costs.** Amharic spends **~170 ms per analysis state** in a cascade already at its +state-count floor. + +### Inside one node (Amharic, 28 words, both probe layers live) + +| bucket | `anCascade` 81,707 ms | `anBattery` 7,368 ms | +| --- | --- | --- | +| clone / freeze / allocation | **46.7%** | 41.9% | +| **remainder (unattributed)** | **37.4%** | 15.4% | +| matcher traversal | 12.5% | **39.3%** | +| FeatureStruct algebra | 3.4% | 3.5% | +| memo-key construction | 0.0% | 0.0% | + +Censuses: 27,385 rule attempts — guards reject 19.5% (application-count 6.3%, unifiability 13.2%), +**80.5% reach matching**. Of 25,178 allomorph pattern matches, **1,938 succeed (7.7%)**, 23,240 +fail (**92.3%**). + +**The headline: only ~16% of the Amharic analysis cascade is linguistic computation.** Pattern +matching plus feature unification is 15.9%. Clone plus unattributed machinery is **84%**. The +engine spends its time constructing and discarding objects, not deciding linguistic questions. + +**Gate outcomes, written before the run.** "Any bucket >=50% becomes the sole build target" — clone +is 46.7%, does not fire. "Pattern matching dominates *and* >=80% of attempts fail -> build a +prefilter" — 92.3% do fail, but matching is 12.5%, so it does not dominate; does not fire. +**"Remainder >=30% means the decomposition is incomplete and that is the result" — 37.4%, FIRES.** + +So there is no declared build target yet. The next probe splits the 37.4%: `RuleCascade` / +enumerator orchestration, trail dictionary operations, LINQ allocation, memo lookups, GC. + +**A caution on the 46.7% before anyone acts on it.** `Word.Clone` being half the cascade does not +by itself mean cloning is waste — the cascade forks a `Word` per rule application precisely so it +can explore alternatives without mutating shared state. The deciding question is what fraction of +clones are *discarded unused*: cloned, rejected by a guard or a failed match, thrown away. Given +92.3% of pattern matches fail, that fraction is plausibly very high, but it is unmeasured. If most +clones are discarded, the lever is **not cloning until a rule commits**. If most survive, the lever +is **making a clone cheaper** (copy-on-write shape, which the RUSTIFY branch already explored). +Different builds; measure first. + +## Two durable code constraints + +Neither is expressible in code, and both bite silently. + +1. **`Word.ReplayOnto` does not splice `_mrulesUnapplied`.** It splices `_mruleApps` and + `_nonHeadApps` only. This is safe *solely* because `AnalysisStateKey` includes the per-rule + count multiset, which guarantees a memo hit has identical arrival and stored-arrival counts. + **Anyone narrowing that key breaks correctness with no test failure.** Fix, if the key ever + changes: store arrival counts on `MemoEntry` and compute `stored - storedArrival + query` — a + no-op under today's key, which is also how to test it. This is the one code change from this + work worth making on its own merits, as defensive hardening. +2. **HermitCrab's morphological rules are not order-invariant.** Same pending-rule multiset, + different application order, different synthesis output — 2 cases on Sena, 12 independently on + Indonesian (e.g. `{meN, -Cont}`). This bounds every packed-readout scheme. It is the assumption + Maxwell and Kaplan (CL 19(4):571–590) rely on and this engine does not satisfy. + +## Calibration record — predictions vs outcomes + +Written before the measurements, scored after. Kept because the misses are informative. + +| prediction | outcome | +| --- | --- | +| Sena key collapse 2–4x | **1.12x** — wrong, and it was the number that decided the project | +| Indonesian key collapse 1.0–1.3x | 1.17x — right | +| Amharic "most likely to disappoint" | best static classification of the three (31 of 36 rules) — wrong | +| Fold sharing ~50% wall ceiling | **0.96x realised** — wrong; the ceiling was measured with a key that would lose parses | +| Amharic synthesis-bound at ~160 ms/run | forward synthesis is **14 ms** — wrong; arithmetic on an unmeasured denominator | + +## Method rules earned here + +1. **Counting is not timing.** Three published findings in this work were retracted, all arithmetic + on an unmeasured denominator: "synthesis is the bottleneck" (candidate counts), "Amharic is + synthesis-bound" (30 s / 186 inputs), "fixtures only look synthesis-heavy because they are small" + (divided by `synForward`, which excludes the buckets the work runs in). +2. **A ratio measured with an incomplete key is not a ratio.** See the table above, three times. +3. **Warm-up dominates small fixtures.** A first A/B here showed 1.53x purely from JIT; the off-arm + alone varied 42.7% between its own first and second sample. Discard warm-up, take min of N, + interleave arms, and print the off-arm spread as a noise floor beside every speedup. +4. **Sub-50 ms fixtures cannot carry a timing claim.** One 10 ms fixture's share moved 1.3% -> 14.5% + between two runs of identical code. +5. **Three grammars minimum, reported unpooled.** Maxwell & Kaplan measured a 100x swing between two + variants of one grammar. A pooled average across fixtures of different size has had to be + retracted twice in this project. +6. **Search completeness is never traded for speed.** HermitCrab is the permanent fallback engine + behind the FST work. +7. **A bucket reporting exactly `0.00` is broken until proven otherwise.** `NodeCostProbe.Enabled` + was declared and read in `Matcher.cs` but set by nothing — two gates in two assemblies, one + wired. Both lower-layer buckets read `0.00ms`, which looks like "measured and negligible" + rather than "never ran", and their time silently inflated the remainder. Wire cross-assembly + probe gates from a single place. +8. **Always carry an explicit remainder column.** It caught the wall-split misattribution and the + dead probe gate. Four of the five retractions in this work would have been caught at the point + of measurement by a remainder that did not sum. From f66b0145701132ad288e03bc9a512402cb35a25e Mon Sep 17 00:00:00 2001 From: John Lambert Date: Thu, 27 Aug 2026 12:07:23 -0400 Subject: [PATCH 2/5] docs: correct four errors and three overreaches found in final review 1. '6 of 7 heavy words exactly 1.00' was FALSE -- the source table shows 4 of 7. The three that moved (1.13, 1.16, 1.38) all return zero parses, so R tracks failure not size, and cinacemerwa -- the corpus's most expensive word -- cleared the 1.3 gate. Row 4's flatness was overstated. Restores the reopening condition and the nogood observation (434,628 vs 25,102), both dropped from the source. 2. The Sena bucket row spliced two runs: battery 51.4% / cascade 18.4% from the 143,303ms run, ExpandAlternatives 20.3% / unaccounted 1.5% from the 112,713ms run. Together they sum past 100%, violating this document's own method rule 8. Now labelled per run and marked not co-summable. 3. 'The engine spends its time constructing and discarding objects, not deciding linguistic questions' overreached -- 37.4% is unattributed, i.e. unknown rather than known-non-linguistic -- and contradicted the fired gate three lines below. 4. Row 20 claimed generation's ratios 'would apply at face value', which is exactly the artifact rows 5/6 closed. Generation changes the share, not the soundness argument, which is workload-independent. Also: scopes the family claim to merge-by-key forests, since row 19's contexted-constraint variant packs WITHOUT merging and the argument does not reach it; softens the Amharic state-floor claim to inferred; downgrades row 14 to adjacent evidence; moves row 21 (CLOSED) into the closed table; and adds gates for the clone-discard fraction, the Sena battery decomposition (potentially ~20% of Sena wall, which would outrank every open row) and the generation probe. Row 16 is flagged as likely understated at 3-8%, since 85% of its duplicates are same-analysis-word and the same-word ceiling is nearer 17%. Co-Authored-By: Claude Opus 5 --- docs/hermitcrab-optimization-ledger.md | 46 ++++++++++++++++---------- 1 file changed, 29 insertions(+), 17 deletions(-) diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md index ca1ccac9..b2954d5c 100644 --- a/docs/hermitcrab-optimization-ledger.md +++ b/docs/hermitcrab-optimization-ledger.md @@ -47,7 +47,7 @@ and this ledger. | # | Optimization | Expected | Why it failed | Number | | --- | --- | --- | --- | --- | -| 4 | **Narrow `AnalysisStateKey`** (drop strictly-shrinking rules' un-application counts) | `MaxApplicationCount` defaults to 1, so the count multiset is "which subset of rules was used" — 2^27 states. Dropping 19 of 27 rules gives 2^8 | Shape + syntactic FS + realizational FS *already* discriminate. The count component was implied by its neighbours. A worst-case bound collapsing is not the realised state count collapsing | Realised state collapse **Sena 1.12x** vs a 1.3 gate; 6 of 7 heavy words exactly **1.00**; `atawirambo` 2,556 -> 2,556 | +| 4 | **Narrow `AnalysisStateKey`** (drop strictly-shrinking rules' un-application counts) | `MaxApplicationCount` defaults to 1, so the count multiset is "which subset of rules was used" — 2^27 states. Dropping 19 of 27 rules gives 2^8 | Shape + syntactic FS + realizational FS *already* discriminate. The count component was implied by its neighbours. A worst-case bound collapsing is not the realised state count collapsing | Realised state collapse **Sena 1.12x** (state-weighted) vs a 1.3 gate; **4 of 7** heavy words exactly 1.00, `atawirambo` 2,556 -> 2,556. The three that moved -- `pidafikawo` 1.13, `cinagumanika` 1.16, **`cinacemerwa` 1.38** -- all return **zero parses**: R tracks failure, not size, and the corpus's single most expensive word *did* clear the gate. **Reopening condition:** a grammar with genuinely high R; the census is cheap and is the right first question to ask of any new grammar | | 5 | **Fold-step sharing memo** (share deterministic synthesis fold steps by computed value) | Probe measured 3.22x and 8.10x shareable steps; ~50% wall ceiling on two reliable fixtures | A sound key must carry the **ordered remaining trail**. Two candidates then have to agree on their entire future to share a step, which almost never happens | **hits = 0** on the best fixture; **0.96x** (4.5% *slower* — pure key-construction cost); 1.06x on the second, inside 21.9% noise | | 6 | **Order-insensitive synthesis-input dedupe** | 9,774x duplicate synthesis inputs | HermitCrab's morphological rules are **not order-invariant**: same rule multiset, different order, different output. Merging loses parses | 2 violations Sena, 12 independently Indonesian (e.g. `{meN, -Cont}`). Sound version: **15–40%** | | 7 | **Trail-position indexing of synthesis rules** | 11,445,538 of 11,445,538 rejections are "this rule is not the pending trail rule" — 100% of the histogram | Each rejection is one array index plus one reference compare. Count is not cost | 11.4M x 29 ns = 0.33 s of 143 s = **0.2%** | @@ -55,6 +55,7 @@ and this ledger. | 9 | Analysis length ceiling (Gate B) | Same idea, analysis side | Sound and correct. No available corpus contains the pathology it prunes | Byte-identical output, **no measurable speedup** (~4–7% slower, within noise) | | 10 | Lexical reachability gate (Phase 5) | Prune candidates from which no lexicon root is reachable | Both reference corpora have real compounding in their deepest stratum, which disqualifies the gate everywhere | Proven **no-op** on both corpora | | 11 | Tandem lexical intersection (T2) | Kill doomed branches early via the lexicon | The oracle only sees lexical unreachability. The expensive words fail on checks that run *after* lexical lookup succeeds | Pooled **23.5%** dead steps vs a 30% gate; one failure word at exactly **0.0%** dead | +| 21 | **Memo-key caching on the frozen `Word`** | **CLOSED.** Keys are built ~28x per distinct state, which looked like a cheap win. Measured cost of all `AnalysisStateKey` construction and hashing on Amharic: **3 ms of 81,707 ms = 0.0%.** Same error shape as #7: high count, no cost | | 12 | Pool small short-lived collections | Fewer allocations, faster | `HashSet/Dictionary.Clear()` is O(capacity); Gen0 already beats pooling at this size | -15–17% bytes but **+8.6% wall**, +12% on the parallel battery. Reverted | ## Closed before building — motivation refuted by measurement @@ -62,20 +63,23 @@ and this ledger. | # | Optimization | Expected | Why it died | | --- | --- | --- | --- | | 13 | Surface-length pruning inside the synthesis fold (P2) | "The only lever on Amharic's ~160 ms per synthesis run" | Amharic's forward synthesis is **14 ms**. The 160 ms was arithmetic on an unmeasured denominator — dividing 30 s by 186 synthesis inputs and assuming the time was there | -| 14 | Nogood lattice subsumption (P4) | Nogood hits dominate real hits 434,628 to 25,102; generalise "empty" over the feature lattice | Shape + features already discriminate almost perfectly (see #4), so subsumption buckets hold one member and nothing generalises | +| 14 | Nogood lattice subsumption (P4) | Nogood hits dominate positive replays **434,628 to 25,102** -- the engine mostly proves subtrees *empty*, and this is the only pointer anywhere at the nogood path | Closed on **adjacent** evidence: #4 measured key equality after projection, not a subsumption ordering over nogood feature structures. The direct probe -- a subsumption census over recorded nogood keys, gate >=30% subsumable -- was never run. Conclusion probably holds; the citation overclaims | | 15 | Move constraints into the analysis key (Maxwell & Kaplan's category-splitting) | Their biggest measured win: make the chart prune what the constraint solver otherwise would | Everything synthesis rejects on is root- or realization-dependent, and the root is unknown until `LexicalLookup`. Also a Sena analysis state costs ~0.73 ms while a synthesis input costs at most ~0.12 ms — **states are dearer than synthesis inputs** | ## Open | # | Optimization | Status | | --- | --- | --- | -| 21 | **Memo-key caching on the frozen `Word`** | **CLOSED.** Keys are built ~28x per distinct state, which looked like a cheap win. Measured cost of all `AnalysisStateKey` construction and hashing on Amharic: **3 ms of 81,707 ms = 0.0%.** Same error shape as #7: high count, no cost | -| 22 | **Analysis-side rule prefilter** (index rules by segment/feature signature so a rule that cannot match is never handed to the matcher) | **Bounded, not built.** 92.3% of allomorph pattern matches fail (23,240 of 25,178) and 80.5% of rule attempts reach matching — so the prefilter target is real. But matcher traversal is only **12.5%** of the Amharic cascade, capping this at ~11.5% there. Worth more on the template battery, where matching is **39.3%** | -| 16 | **`ExpandAlternatives` dedupe** | Live. 20.3% of Sena wall time, never instrumented before this work. 926 alternatives -> 3 distinct on one fixture, **100% same-analysis-word** so interceptable pre-expansion. Sound ceiling **3–8% Sena**. Needs a trail-complete key | +| 22 | **Analysis-side rule prefilter** (index rules by segment/feature signature so a rule that cannot match is never handed to the matcher) | **Bounded, not built.** 92.3% of allomorph pattern matches fail (23,240 of 25,178) and 80.5% of rule attempts reach matching — so the prefilter target is real. But matcher traversal is only **12.5%** of the Amharic cascade, capping this at ~11.5% there. Worth more on the template battery, where matching is **39.3%** -- but that is the *Amharic* +battery, which is only ~8% of its wall. **Sena's battery is 51.4% of its wall and has never been +decomposed per-node.** If its match share is comparable, the Sena cap is ~0.514 x 0.39 = **~20% of +wall**, outranking every other open row. Probe: `NodeCostProbe` over the 3 Sena heavy words; gate -- +matcher >=30% of Sena `anBattery` makes this the top open lead | +| 16 | **`ExpandAlternatives` dedupe** | Live, and probably the strongest open lead. 20.3% of Sena wall, never instrumented before this work. 926 alternatives -> 3 distinct on one fixture; **85% of duplicates pooled (100% on two words) trace to the same analysis word**, so they share a trail, so the existing key's omission is harmless *for those*. The 3-8% figure applies the cross-word sound rate (15-40%) to all of them and therefore likely **understates it -- the same-word ceiling is nearer 17%**. Run the sound same-word census before quoting 3-8% | | 17 | **Per-node cost in the analysis cascade** | Live and unexplored — see below | | 18 | Corpus-scope memoization (P3) | Not run. `AnalysisScope` dies per word but the key is word-independent by construction. Amharic's corpus run takes 4.3 hours | | 19 | Contexted constraints / abstract feature-only replay | Not built. Gated on a die-point histogram that says candidates die on shape-free checks | -| 20 | Generation (`GenerateWords`) | **Unmeasured.** Pure synthesis, no analysis phase, so share ~100% — the fold-step ratios would apply at face value. Every verdict above is scoped to *parsing* | +| 20 | Generation (`GenerateWords`) | **Unmeasured.** Pure synthesis, so the *share* is ~100% rather than ~5%. That changes the share, **not** the soundness argument: rows 5 and 6 fail because a trail-incomplete key is unsound, which is workload-independent. Do **not** read this row as "the ratios apply at face value" -- that is the artifact rows 5/6 closed. Every verdict above is scoped to **parsing**. Probe: run the existing off-by-default fold memo over the 33 fixtures in generation; gate -- hit rate >=5% of steps reopens row 5 for generation, 0 closes the family there too | --- @@ -93,22 +97,27 @@ Three independent measurements, three boundaries, same collapse: step distinct, and every measurement showing large shareable work is measuring a key that omits it.** The thing that makes sharing visible is the thing that makes sharing wrong. -That closes a *family*: packed parse forests, fold-step sharing, and synthesis-input dedupe all -require distinct derivations to converge on a genuinely identical state. In this engine they do -not converge. It is the same fact as the rules being non-order-invariant, seen from the other side. +That closes a *family*: packed parse forests **of the merge-by-key kind**, fold-step sharing and +synthesis-input dedupe all require distinct derivations to converge on a genuinely identical +state. In this engine they do not converge -- the same fact as the rules being non-order-invariant, +seen from the other side. **It does not close row 19**: a contexted-constraint forest packs +*without* merging states, so this argument does not reach it. Row 19 stays open and unmeasured. ## Where the time actually is -Measured with eight exclusive buckets summing to wall (`unaccounted` 0.1% on Amharic, 1.5% on Sena): +Measured with eight exclusive buckets summing to wall. Amharic is a single run (`unaccounted` +0.1%). **Sena is spliced from two runs -- see the note in its row.** | grammar | breakdown | | --- | --- | -| **Sena** heavy words | analysis template battery **51.4%** (still, after the 5x memo), `ExpandAlternatives` **20.3%**, analysis cascade 18.4%, **all synthesis ~5%** | +| **Sena** heavy words | battery **51.4%**, cascade 18.4%, all synthesis ~5% -- *pre-`ExpandAlternatives` run (143,303 ms, unaccounted 20.1%)*; `ExpandAlternatives` **20.3%**, unaccounted 1.5% -- *later run (112,713 ms)*. **Two runs: not co-summable.** One re-run would put all Sena shares on one denominator | | **Amharic** | analysis cascade **~95%** of analysis, which is 99.5% of wall; **all synthesis 0.3%** | **Every optimization in this ledger that worked reduced how many nodes are visited. None touched -what a node costs.** Amharic spends **~170 ms per analysis state** in a cascade already at its -state-count floor. +what a node costs.** Amharic spends **~170 ms per analysis state**. (The *measured* state-count floor -- 2,555 +expansions against 2,546 states -- is Sena `atawirambo`. For Amharic, near-floor behaviour is +**inferred** from R ~ 1, which compares the full key against the narrowed key, not against a +semantic floor.) ### Inside one node (Amharic, 28 words, both probe layers live) @@ -124,9 +133,11 @@ Censuses: 27,385 rule attempts — guards reject 19.5% (application-count 6.3%, **80.5% reach matching**. Of 25,178 allomorph pattern matches, **1,938 succeed (7.7%)**, 23,240 fail (**92.3%**). -**The headline: only ~16% of the Amharic analysis cascade is linguistic computation.** Pattern -matching plus feature unification is 15.9%. Clone plus unattributed machinery is **84%**. The -engine spends its time constructing and discarding objects, not deciding linguistic questions. +**The headline: only ~16% of the Amharic analysis cascade is *identified* as linguistic +computation** -- matching 12.5% plus feature unification 3.4%. Clone/freeze is **46.7%** and +**37.4% is unattributed**, which is unknown rather than known-non-linguistic. The defensible +statement is that at most a sixth of the cascade has been shown to decide linguistic questions; +of the rest, half is measured (clone) and half is still open. **Gate outcomes, written before the run.** "Any bucket >=50% becomes the sole build target" — clone is 46.7%, does not fire. "Pattern matching dominates *and* >=80% of attempts fail -> build a @@ -143,7 +154,8 @@ clones are *discarded unused*: cloned, rejected by a guard or a failed match, th 92.3% of pattern matches fail, that fraction is plausibly very high, but it is unmeasured. If most clones are discarded, the lever is **not cloning until a rule commits**. If most survive, the lever is **making a clone cheaper** (copy-on-write shape, which the RUSTIFY branch already explored). -Different builds; measure first. +Different builds; measure first. **Gate: probe the clone-discard fraction; >=70% discarded means +the defer-clone build, below that the cheaper-clone build.** ## Two durable code constraints From bc80bf0b01932a7d06a7c83160a556159819d4c6 Mon Sep 17 00:00:00 2001 From: John Lambert Date: Thu, 27 Aug 2026 15:04:50 -0400 Subject: [PATCH 3/5] docs: fold in all seven probe results -- 6 rows closed, 1 partly open Consolidates rows 14, 16, 17, 18, 19, 20 and 22 from the six probe branches that measured them. Net result of the round: no optimization survived, and the ledger now stands at 21 closed / 1 partly open out of 22. Highlights: - Row 18 is worse than a no-op, not neutral: Sena 300 words runs 28-32% SLOWER under a shared scope with ~29.3 GB peak, because MaxMemoEntries is a per-SCOPE cap -- two heavy early words exhaust it and every later word then loses the within-word memoization a fresh scope would have given it. Cross-word hits are negative (-77,648 memo, -747,607 nogood). - Row 16's sound census: 395,026 alternatives -> 351,414 distinct (89.0%), ceiling 2.3-2.5%. Two of three Sena words have zero real duplicates. - Row 14: every nogood bucket is a singleton, 0.0% subsumable, across Sena and all 33 fixtures. - Row 19: shape-free die-point cost 0.0-6.0%, and total rejection cost is 0.2% of wall -- two independent reasons to close. - Row 22: Sena's battery does not understate the prefilter cap; 21.2% matcher share gives ~10.1%, the same bound as Amharic's. - Row 20: generation hits 2.67% pooled, 37.7% on its best fixture, and still loses to noise. - Row 17 remains partly open: the ~38% cascade remainder is unattributed, Equality and Memo and Trail each measured <0.2%, and GC pause time is the leading candidate but netstandard2.0 cannot measure it. Zero duplicate tokens on every corpus including full Sena (7,121) and Amharic (673), so type-level caching was never available either. Co-Authored-By: Claude Opus 5 --- docs/hermitcrab-optimization-ledger.md | 115 +++++-------------------- 1 file changed, 23 insertions(+), 92 deletions(-) diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md index b2954d5c..6ea2fccb 100644 --- a/docs/hermitcrab-optimization-ledger.md +++ b/docs/hermitcrab-optimization-ledger.md @@ -8,30 +8,9 @@ already been independently rediscovered and re-retracted. Deliberately in-repo and durable, not evicted into a PR accordion. A closed avenue is only closed if the next person can find out cheaply that it is closed. -**This is the only report.** The working docs it was distilled from -(`hermitcrab-packed-forest-research.md`, `hermitcrab-forest-memo-plan.md`, -`hermitcrab-forest-memo-ceiling.md`, `hermitcrab-synthesis-fold-probes.md`) are chronological -process records containing retracted intermediate claims, and are deliberately **not** merged. -They stay on the branches below, together with all instrumentation, as reproducible evidence: - -| branch | holds | -| --- | --- | -| `feature/forest-memo` | key-narrowing measurement, `RuleLengthClassifier` + 12 tests | -| `feature/synthesis-fold-probes` | the eight-bucket wall-time split, die-point histogram, sharing census, harness | -| `feature/synthesis-fold-sharing` | `SynthesisStateKey` + fold memo, parity-clean, `hits = 0` | -| `feature/per-node-cost` | `NodeCostProbe` (SIL.Machine layer) + the per-node decomposition | - -**No instrumentation is merged, by design.** The probes gate on `volatile bool` reads inside -`Matcher.cs` — a hot inner loop in `SIL.Machine` used by every consumer — which is an acceptable -cost for a measurement run and not one to carry in product code indefinitely. Rebuilding a probe -from a branch is cheap; maintaining research scaffolding in two product assemblies is not. - -## Net speed delivered: none - -No optimization in the "closed" section shipped, and none of them made anything faster. The -fastest thing tried was 4% **slower**. The pre-existing wins in this codebase (rows 1–3) predate -this work. What this effort produced is the measurement, the closed family, the located target, -and this ledger. +Sources: `hermitcrab-packed-forest-research.md` (theory + prior branches), +`hermitcrab-forest-memo-plan.md` (key narrowing), `hermitcrab-forest-memo-ceiling.md` +(predictions), `hermitcrab-synthesis-fold-probes.md` (measurements, sections 6–10). --- @@ -47,7 +26,7 @@ and this ledger. | # | Optimization | Expected | Why it failed | Number | | --- | --- | --- | --- | --- | -| 4 | **Narrow `AnalysisStateKey`** (drop strictly-shrinking rules' un-application counts) | `MaxApplicationCount` defaults to 1, so the count multiset is "which subset of rules was used" — 2^27 states. Dropping 19 of 27 rules gives 2^8 | Shape + syntactic FS + realizational FS *already* discriminate. The count component was implied by its neighbours. A worst-case bound collapsing is not the realised state count collapsing | Realised state collapse **Sena 1.12x** (state-weighted) vs a 1.3 gate; **4 of 7** heavy words exactly 1.00, `atawirambo` 2,556 -> 2,556. The three that moved -- `pidafikawo` 1.13, `cinagumanika` 1.16, **`cinacemerwa` 1.38** -- all return **zero parses**: R tracks failure, not size, and the corpus's single most expensive word *did* clear the gate. **Reopening condition:** a grammar with genuinely high R; the census is cheap and is the right first question to ask of any new grammar | +| 4 | **Narrow `AnalysisStateKey`** (drop strictly-shrinking rules' un-application counts) | `MaxApplicationCount` defaults to 1, so the count multiset is "which subset of rules was used" — 2^27 states. Dropping 19 of 27 rules gives 2^8 | Shape + syntactic FS + realizational FS *already* discriminate. The count component was implied by its neighbours. A worst-case bound collapsing is not the realised state count collapsing | Realised state collapse **Sena 1.12x** vs a 1.3 gate; 6 of 7 heavy words exactly **1.00**; `atawirambo` 2,556 -> 2,556 | | 5 | **Fold-step sharing memo** (share deterministic synthesis fold steps by computed value) | Probe measured 3.22x and 8.10x shareable steps; ~50% wall ceiling on two reliable fixtures | A sound key must carry the **ordered remaining trail**. Two candidates then have to agree on their entire future to share a step, which almost never happens | **hits = 0** on the best fixture; **0.96x** (4.5% *slower* — pure key-construction cost); 1.06x on the second, inside 21.9% noise | | 6 | **Order-insensitive synthesis-input dedupe** | 9,774x duplicate synthesis inputs | HermitCrab's morphological rules are **not order-invariant**: same rule multiset, different order, different output. Merging loses parses | 2 violations Sena, 12 independently Indonesian (e.g. `{meN, -Cont}`). Sound version: **15–40%** | | 7 | **Trail-position indexing of synthesis rules** | 11,445,538 of 11,445,538 rejections are "this rule is not the pending trail rule" — 100% of the histogram | Each rejection is one array index plus one reference compare. Count is not cost | 11.4M x 29 ns = 0.33 s of 143 s = **0.2%** | @@ -55,7 +34,6 @@ and this ledger. | 9 | Analysis length ceiling (Gate B) | Same idea, analysis side | Sound and correct. No available corpus contains the pathology it prunes | Byte-identical output, **no measurable speedup** (~4–7% slower, within noise) | | 10 | Lexical reachability gate (Phase 5) | Prune candidates from which no lexicon root is reachable | Both reference corpora have real compounding in their deepest stratum, which disqualifies the gate everywhere | Proven **no-op** on both corpora | | 11 | Tandem lexical intersection (T2) | Kill doomed branches early via the lexicon | The oracle only sees lexical unreachability. The expensive words fail on checks that run *after* lexical lookup succeeds | Pooled **23.5%** dead steps vs a 30% gate; one failure word at exactly **0.0%** dead | -| 21 | **Memo-key caching on the frozen `Word`** | **CLOSED.** Keys are built ~28x per distinct state, which looked like a cheap win. Measured cost of all `AnalysisStateKey` construction and hashing on Amharic: **3 ms of 81,707 ms = 0.0%.** Same error shape as #7: high count, no cost | | 12 | Pool small short-lived collections | Fewer allocations, faster | `HashSet/Dictionary.Clear()` is O(capacity); Gen0 already beats pooling at this size | -15–17% bytes but **+8.6% wall**, +12% on the parallel battery. Reverted | ## Closed before building — motivation refuted by measurement @@ -63,23 +41,20 @@ and this ledger. | # | Optimization | Expected | Why it died | | --- | --- | --- | --- | | 13 | Surface-length pruning inside the synthesis fold (P2) | "The only lever on Amharic's ~160 ms per synthesis run" | Amharic's forward synthesis is **14 ms**. The 160 ms was arithmetic on an unmeasured denominator — dividing 30 s by 186 synthesis inputs and assuming the time was there | -| 14 | Nogood lattice subsumption (P4) | Nogood hits dominate positive replays **434,628 to 25,102** -- the engine mostly proves subtrees *empty*, and this is the only pointer anywhere at the nogood path | Closed on **adjacent** evidence: #4 measured key equality after projection, not a subsumption ordering over nogood feature structures. The direct probe -- a subsumption census over recorded nogood keys, gate >=30% subsumable -- was never run. Conclusion probably holds; the citation overclaims | +| 14 | Nogood lattice subsumption (P4) | Nogood hits dominate positive replays **434,628 to 25,102** — the engine mostly proves subtrees *empty*, and generalising "empty" over the feature lattice would turn nogoods from points into regions | **CLOSED on direct evidence** (it was previously closed on adjacent evidence, which is why it was reopened). Census over every empty-expansion key: `cinacemerwa` 16,176 expansions -> 16,176 distinct keys -> **16,176 singleton buckets, 0 subsumable (0.0%)**; `kukucitirani` 12,595 -> 12,595 -> **12,595 singletons, 0 subsumable**. Not one pair of nogood entries even shares a `(shape, stratum, counts, non-head)` bucket, so there is nothing for a subsumption ordering to act on. Breadth check across all 33 conformance fixtures agrees exactly: 7,493 empty expansions, 7,398 distinct keys, **0 subsumable (0.0%)** pooled, every non-empty bucket a singleton. Direction checked explicitly as `earlier.Subsumes(later)` (E subsumes Q, general over specific); the reverse would be unsound. Independently reconfirms row 4 from a different angle: the keys are maximally discriminating | | 15 | Move constraints into the analysis key (Maxwell & Kaplan's category-splitting) | Their biggest measured win: make the chart prune what the constraint solver otherwise would | Everything synthesis rejects on is root- or realization-dependent, and the root is unknown until `LexicalLookup`. Also a Sena analysis state costs ~0.73 ms while a synthesis input costs at most ~0.12 ms — **states are dearer than synthesis inputs** | ## Open | # | Optimization | Status | | --- | --- | --- | -| 22 | **Analysis-side rule prefilter** (index rules by segment/feature signature so a rule that cannot match is never handed to the matcher) | **Bounded, not built.** 92.3% of allomorph pattern matches fail (23,240 of 25,178) and 80.5% of rule attempts reach matching — so the prefilter target is real. But matcher traversal is only **12.5%** of the Amharic cascade, capping this at ~11.5% there. Worth more on the template battery, where matching is **39.3%** -- but that is the *Amharic* -battery, which is only ~8% of its wall. **Sena's battery is 51.4% of its wall and has never been -decomposed per-node.** If its match share is comparable, the Sena cap is ~0.514 x 0.39 = **~20% of -wall**, outranking every other open row. Probe: `NodeCostProbe` over the 3 Sena heavy words; gate -- -matcher >=30% of Sena `anBattery` makes this the top open lead | -| 16 | **`ExpandAlternatives` dedupe** | Live, and probably the strongest open lead. 20.3% of Sena wall, never instrumented before this work. 926 alternatives -> 3 distinct on one fixture; **85% of duplicates pooled (100% on two words) trace to the same analysis word**, so they share a trail, so the existing key's omission is harmless *for those*. The 3-8% figure applies the cross-word sound rate (15-40%) to all of them and therefore likely **understates it -- the same-word ceiling is nearer 17%**. Run the sound same-word census before quoting 3-8% | -| 17 | **Per-node cost in the analysis cascade** | Live and unexplored — see below | -| 18 | Corpus-scope memoization (P3) | Not run. `AnalysisScope` dies per word but the key is word-independent by construction. Amharic's corpus run takes 4.3 hours | -| 19 | Contexted constraints / abstract feature-only replay | Not built. Gated on a die-point histogram that says candidates die on shape-free checks | -| 20 | Generation (`GenerateWords`) | **Unmeasured.** Pure synthesis, so the *share* is ~100% rather than ~5%. That changes the share, **not** the soundness argument: rows 5 and 6 fail because a trail-incomplete key is unsound, which is workload-independent. Do **not** read this row as "the ratios apply at face value" -- that is the artifact rows 5/6 closed. Every verdict above is scoped to **parsing**. Probe: run the existing off-by-default fold memo over the 33 fixtures in generation; gate -- hit rate >=5% of steps reopens row 5 for generation, 0 closes the family there too | +| 21 | **Memo-key caching on the frozen `Word`** | **CLOSED.** Keys are built ~28x per distinct state, which looked like a cheap win. Measured cost of all `AnalysisStateKey` construction and hashing on Amharic: **3 ms of 81,707 ms = 0.0%.** Same error shape as #7: high count, no cost | +| 22 | **Analysis-side rule prefilter** | 94.4% of Sena allomorph pattern matches fail (6,609,627 of 7,005,273) and 86.7% of 4,327,182 rule attempts reach matching, so the target is real | **CLOSED, cap ~10%.** The reopening hypothesis was that Sena's battery would badly understate the Amharic-derived ~11.5% bound. **It does not.** Sena `anBattery` matcher share is **21.2%**, against a >=30% gate — does not fire — giving a cap of 47.6% x 21.2% = **~10.1%**, essentially the same bound. Contended run; the gap to the gate is wide enough that contention will not close it | +| 16 | **`ExpandAlternatives` dedupe** | 20.7% of Sena wall, the largest bucket nobody had instrumented. Apparent census: 395,026 alternatives -> **61** distinct (0.0%), 85% of duplicates same-analysis-word | **CLOSED on the sound census.** With the pending-trail content added to the key, 395,026 -> **351,414 distinct (89.0%)** -- only **11%** are duplicates at all, and all 43,612 of them are same-analysis-word. Sound ceiling = 20.7% x 0.110 = **2.28% of Sena wall**, against a gate of >=10% to build and <5% to close. **The fourth apparent redundancy to evaporate under a complete key** | +| 17 | **Per-node cost in the analysis cascade** | 84% of the Amharic cascade is not linguistic computation; clone is 46.7% and 37.4% was unattributed | **Partly settled, remainder still open.** (a) The remainder split **MISSES its <15% gate by ~2.5x** — still ~38% on Amharic `anCascade`. Memo, Trail and **Equality each measured <0.2%**, so `Word.ValueEquals` behind the cascade's `HashSet` is *not* the missing mass, contrary to the standing hypothesis. Leading unbracketed candidate is **GC pause time**, which netstandard2.0 cannot measure directly (`GC.GetTotalPauseDuration` is .NET 7+); a collection census over 24 Amharic words gives gen0 8,217 / gen1 2,729 / gen2 301, an unusually high gen1+gen2 ratio consistent with — not proof of — GC as the remainder. (b) **Clone-discard fraction = 66.2%** (created 2,505, discarded 1,659), a count and therefore contention-immune, against a >=70% gate: **misses by ~4 points, so the build is "make a clone cheaper", not "defer cloning"**. Scoped to `AnalysisAffixTemplateRule`'s eager per-template clone, the one clone-before-verification site; the other four analysis-side sites clone only after a successful match and were excluded rather than blended in | +| 18 | Corpus-scope memoization | `AnalysisScope` dies per word, but `AnalysisStateKey` is word-independent by construction, so entries are reusable across a corpus. Nogood hits dominate positive replays 10-17x, making cross-word nogood reuse the cheapest available form of reuse | **CLOSED, and worse than a no-op.** Indonesian, 121 words, **serial on a clean machine: 1.015x, 1.4% wall-clock reduction** against a >=20% build gate and <5% close gate. Costs **68 MB** added peak managed memory. Cross-word reuse is real but small: 39 memo, 73 nogood, 237 template-nogood hits. Token-vs-type: **121 tokens, 121 distinct**, so no type-level dedupe is available either. **A first, contended run of the same harness reported 2.385x / 58.1%** with a 210.9% spread on the shared arm against 13.7% on the baseline; two testhosts were live. The counts were byte-identical across both runs — only the timing was contaminated, and min-of-N on a high-spread arm picks the luckiest sample. Analysis-set equality held throughout, so the key genuinely is word-independent. **Sena, first 300 words, is the decisive case: the shared scope is 28-32% *slower* than per-word scopes** (two clean serial runs, no contention: 282.7s vs 372.2s and 308.9s vs 396.7s baseline-vs-shared; deterministic counters byte-identical across both runs), and costs **~29.3 GB** added peak managed memory against a 2 GB budget. Root cause: `AnalysisScope.MaxMemoEntries` (100,000) is a *per-scope*, not per-word, budget. Two heavy early words (#74, #76 of 300) alone fill the shared `Memo` table to the cap and `TemplateMemo` to 55,071; every word after that loses the free within-word memoization a fresh per-word scope would have given it for nothing, because `Store` silently no-ops past the cap (see its comment) with no eviction. Cross-word hits (shared minus baseline) are **negative** — memo -77,648, nogood -747,607 — i.e. sharing produced *fewer* total hits than 300 independent fresh-table runs would have on their own; the small `+820` template-hit / `+9` template-nogood gain does not come close to covering it. Amharic, capped at 28 words (word 29 hangs, excluded per standing guidance): **0.9% wall-clock reduction**, table nowhere near the cap (Memo 1,251, TemplateMemo 652 of 100,000), consistent with row 17's finding that Amharic's cost is dominated by clone/allocation per node rather than distinct-state count, so avoiding a handful of cross-word re-derivations barely moves wall time. Token-vs-type on all three corpora, at the runs' own sizes and on each corpus's full word list: **zero duplicate tokens** — whole-word caching is not available as a cheaper alternative anywhere here. No analysis-set divergence on any corpus, at any scope regime: the stop condition that would have put rows 1-3 in question did not fire. **A shared-but-capped memo table is not "the same optimization, smaller ceiling" — past the cap it actively takes away the free per-word memoization the product already has, and retains every live entry for the run's full duration instead of per-word, which is where the 29 GB comes from.** Any future attempt needs real eviction (LRU, generational, or per-word sub-budgets) before this is worth re-measuring, and even then the >=20% gate looks unreachable given Indonesian's ceiling under an uncapped-in-practice regime was already only 1.4% | +| 19 | Contexted constraints / abstract feature-only replay | **CLOSED.** Cost-weighted the P1b die-point histogram (count → wall-time per die point, gated before the run: shape-free cost ≥60% builds, <30% closes). Sena `atawirambo` **6.0%**, `cinacemerwa` **0.0%**, Amharic 28 words **0.4%** — all fire the close gate, none come close to the middle band. All 33 conformance fixtures measured too (breadth, not part of the gate): 24 close, 3 fire the build gate (`morphotactic-attribute-breadth` 67.1%, `mpr-group-overwrite-without-realizational` 91.5%, `prefixal-discontinuous-slot-dependency` 71.8%), 3 land in the undecided 30–60% band, 2 have zero rejections, 1 has zero words. **All three build-gate fixtures are sub-50 ms** (38.15 ms / 1.49 ms / 4.28 ms wall) — the method-rules-earned "cannot carry a timing claim" flag, so they do not override the real-corpus verdict. On the three real grammars, count and cost agree in direction (`RuleNotApplicableOrPatternMismatch`, shape-dependent, dominates both). Among the fixtures, one reliable one (wall ≥50 ms) shows the count/cost divergence the gate exists to catch: `suffixing-extension-slot-ordering` (53 words, 63.98 ms) shape-free count 12.9% vs. cost 53.3% — cost roughly 4x count, same direction as the ApplicationCount/MprFeatures/FeatureUnification checks costing more per event than pattern-mismatch's near-O(1) trail check. (`feature-gating-breadth`, 28.6% vs. 56.8%, shows the same direction but is sub-50 ms and unreliable, same flag as the three build-gate fixtures.) Orthogonal to classification: die-point *rejection* cost itself is negligible against wall time on all three real-corpus runs regardless of class (atawirambo ~19.8 ms of 11,547 ms wall = 0.17%; cinacemerwa ~112.9 ms of 57,275 ms = 0.20%; Amharic ~88.8 ms of 91,456 ms = 0.10%) — reconfirming row 7/6.2's finding that rejections are cheap. Abstract replay would be optimizing a phase that costs approximately nothing on the two grammars this project treats as ground truth. Not built | +| 20 | **Row 5's family on `GenerateWords`** (generation, ~100% synthesis share vs parsing's ~5%/~0.3%) | Row 20 was unmeasured; generation's share is ~100% so row 5's fold ratios would apply at face value if they transferred. `UseSynthesisFoldMemo` had never been wired into `GenerateWords` (only into the parse-then-synthesize path) — wired in under the same untraced/`MaxDegreeOfParallelism==1` restriction to make this measurable at all | Hits are no longer uniformly zero: 3 of 23 lookup-reaching fixtures clear the 5% gate (diacritic-segments 33.3%, suffixing-vowel-harmony 28.6%, suffixing-evidential-adjacency-chain 37.7%; a 4th sits just under it at 4.8%), reopening row 5 narrowly for generation. But the reopened case recloses immediately: the A/B on the best fixture shows the wall-clock "win" sitting inside its own noise floor, and the pooled hit rate is dominated by `deep-optional-affix-nesting` (925 of 1,613 lookups) at hits=0, reproducing row 5's exact 4.5%-slower key-construction cost on the identical fixture | Parity: 0 divergences, 1,236 generations, 32/33 fixtures (33rd throws identically both sides). Best case: 130 hits / 5 reps (37.7% hit rate), **7.8%** wall-clock reduction inside a **22.2%** off-arm spread — not a result. 19/23 lookup-reaching fixtures at exactly **0%**; pooled hit rate **2.67%** (43/1,613), skewed by the one large zero-hit fixture | --- @@ -97,27 +72,22 @@ Three independent measurements, three boundaries, same collapse: step distinct, and every measurement showing large shareable work is measuring a key that omits it.** The thing that makes sharing visible is the thing that makes sharing wrong. -That closes a *family*: packed parse forests **of the merge-by-key kind**, fold-step sharing and -synthesis-input dedupe all require distinct derivations to converge on a genuinely identical -state. In this engine they do not converge -- the same fact as the rules being non-order-invariant, -seen from the other side. **It does not close row 19**: a contexted-constraint forest packs -*without* merging states, so this argument does not reach it. Row 19 stays open and unmeasured. +That closes a *family*: packed parse forests, fold-step sharing, and synthesis-input dedupe all +require distinct derivations to converge on a genuinely identical state. In this engine they do +not converge. It is the same fact as the rules being non-order-invariant, seen from the other side. ## Where the time actually is -Measured with eight exclusive buckets summing to wall. Amharic is a single run (`unaccounted` -0.1%). **Sena is spliced from two runs -- see the note in its row.** +Measured with eight exclusive buckets summing to wall (`unaccounted` 0.1% on Amharic, 1.5% on Sena): | grammar | breakdown | | --- | --- | -| **Sena** heavy words | battery **51.4%**, cascade 18.4%, all synthesis ~5% -- *pre-`ExpandAlternatives` run (143,303 ms, unaccounted 20.1%)*; `ExpandAlternatives` **20.3%**, unaccounted 1.5% -- *later run (112,713 ms)*. **Two runs: not co-summable.** One re-run would put all Sena shares on one denominator | +| **Sena** heavy words | analysis template battery **51.4%** (still, after the 5x memo), `ExpandAlternatives` **20.3%**, analysis cascade 18.4%, **all synthesis ~5%** | | **Amharic** | analysis cascade **~95%** of analysis, which is 99.5% of wall; **all synthesis 0.3%** | **Every optimization in this ledger that worked reduced how many nodes are visited. None touched -what a node costs.** Amharic spends **~170 ms per analysis state**. (The *measured* state-count floor -- 2,555 -expansions against 2,546 states -- is Sena `atawirambo`. For Amharic, near-floor behaviour is -**inferred** from R ~ 1, which compares the full key against the narrowed key, not against a -semantic floor.) +what a node costs.** Amharic spends **~170 ms per analysis state** in a cascade already at its +state-count floor. ### Inside one node (Amharic, 28 words, both probe layers live) @@ -133,11 +103,9 @@ Censuses: 27,385 rule attempts — guards reject 19.5% (application-count 6.3%, **80.5% reach matching**. Of 25,178 allomorph pattern matches, **1,938 succeed (7.7%)**, 23,240 fail (**92.3%**). -**The headline: only ~16% of the Amharic analysis cascade is *identified* as linguistic -computation** -- matching 12.5% plus feature unification 3.4%. Clone/freeze is **46.7%** and -**37.4% is unattributed**, which is unknown rather than known-non-linguistic. The defensible -statement is that at most a sixth of the cascade has been shown to decide linguistic questions; -of the rest, half is measured (clone) and half is still open. +**The headline: only ~16% of the Amharic analysis cascade is linguistic computation.** Pattern +matching plus feature unification is 15.9%. Clone plus unattributed machinery is **84%**. The +engine spends its time constructing and discarding objects, not deciding linguistic questions. **Gate outcomes, written before the run.** "Any bucket >=50% becomes the sole build target" — clone is 46.7%, does not fire. "Pattern matching dominates *and* >=80% of attempts fail -> build a @@ -154,36 +122,7 @@ clones are *discarded unused*: cloned, rejected by a guard or a failed match, th 92.3% of pattern matches fail, that fraction is plausibly very high, but it is unmeasured. If most clones are discarded, the lever is **not cloning until a rule commits**. If most survive, the lever is **making a clone cheaper** (copy-on-write shape, which the RUSTIFY branch already explored). -Different builds; measure first. **Gate: probe the clone-discard fraction; >=70% discarded means -the defer-clone build, below that the cheaper-clone build.** - -## Two durable code constraints - -Neither is expressible in code, and both bite silently. - -1. **`Word.ReplayOnto` does not splice `_mrulesUnapplied`.** It splices `_mruleApps` and - `_nonHeadApps` only. This is safe *solely* because `AnalysisStateKey` includes the per-rule - count multiset, which guarantees a memo hit has identical arrival and stored-arrival counts. - **Anyone narrowing that key breaks correctness with no test failure.** Fix, if the key ever - changes: store arrival counts on `MemoEntry` and compute `stored - storedArrival + query` — a - no-op under today's key, which is also how to test it. This is the one code change from this - work worth making on its own merits, as defensive hardening. -2. **HermitCrab's morphological rules are not order-invariant.** Same pending-rule multiset, - different application order, different synthesis output — 2 cases on Sena, 12 independently on - Indonesian (e.g. `{meN, -Cont}`). This bounds every packed-readout scheme. It is the assumption - Maxwell and Kaplan (CL 19(4):571–590) rely on and this engine does not satisfy. - -## Calibration record — predictions vs outcomes - -Written before the measurements, scored after. Kept because the misses are informative. - -| prediction | outcome | -| --- | --- | -| Sena key collapse 2–4x | **1.12x** — wrong, and it was the number that decided the project | -| Indonesian key collapse 1.0–1.3x | 1.17x — right | -| Amharic "most likely to disappoint" | best static classification of the three (31 of 36 rules) — wrong | -| Fold sharing ~50% wall ceiling | **0.96x realised** — wrong; the ceiling was measured with a key that would lose parses | -| Amharic synthesis-bound at ~160 ms/run | forward synthesis is **14 ms** — wrong; arithmetic on an unmeasured denominator | +Different builds; measure first. ## Method rules earned here @@ -202,11 +141,3 @@ Written before the measurements, scored after. Kept because the misses are infor retracted twice in this project. 6. **Search completeness is never traded for speed.** HermitCrab is the permanent fallback engine behind the FST work. -7. **A bucket reporting exactly `0.00` is broken until proven otherwise.** `NodeCostProbe.Enabled` - was declared and read in `Matcher.cs` but set by nothing — two gates in two assemblies, one - wired. Both lower-layer buckets read `0.00ms`, which looks like "measured and negligible" - rather than "never ran", and their time silently inflated the remainder. Wire cross-assembly - probe gates from a single place. -8. **Always carry an explicit remainder column.** It caught the wall-split misattribution and the - dead probe gate. Four of the five retractions in this work would have been caught at the point - of measurement by a remainder that did not sum. From 9c34bb9d2bf82ed3501430895a8c1e625f1ba20f Mon Sep 17 00:00:00 2001 From: John Lambert Date: Thu, 27 Aug 2026 15:15:09 -0400 Subject: [PATCH 4/5] docs: rebuild the consolidation on the CORRECTED base, not the probe base The previous fold-in (bc80bf0b) was built from feature/per-node-cost, which predates the corrections made in the prior review round. It silently reverted them and dropped ~93 lines (213 -> 144). Rebuilt as: corrected base f66b0145 + the seven verified row cells + this round's additions. Restored, all of which bc80bf0b had deleted: the '4 of 7 heavy words' correction (bc80bf0b had regressed to the false '6 of 7'); the Sena two-run splice disclosure ('not co-summable'); the two durable code constraints, including the Word.ReplayOnto / _mrulesUnapplied hazard that the PR body advertises; the 'net speed delivered: none' section; the calibration record; and the 'merge-by-key' family qualifier. Added this round: method rules 6-9 as a union rather than one branch's subset (min-of-N vs spread; never dispatch timing probes concurrently; a shared bounded cache without eviction can be worse than no sharing; never aggregate the test logger's doubled output). Corrected two things in row 17(a). The ~38% remainder came from a CONTENDED run, and descheduled wall time lands preferentially in the remainder while deflating self-time shares -- CPU starvation is an unbracketed candidate biasing exactly that way. And 'netstandard2.0 cannot measure GC pause' is wrong at the measurement point: only the libraries are netstandard2.0, while the probe harness targets net10.0, so GC.GetTotalPauseDuration() is directly callable. Gated two-step probe recorded. Also fixed an internal contradiction (the clone caution still called the discard fraction unmeasured after row 17b measured it at 66.2%), added row 22's bucket-exclusivity caveat, retitled the Open section now that it holds outcomes, and extended the evidence table to all seven probe branches -- which are now pushed, since they were local-only and one worktree prune from gone. Co-Authored-By: Claude Opus 5 --- docs/hermitcrab-optimization-ledger.md | 173 +++++++++++++++++++------ 1 file changed, 131 insertions(+), 42 deletions(-) diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md index 6ea2fccb..27ec0bb2 100644 --- a/docs/hermitcrab-optimization-ledger.md +++ b/docs/hermitcrab-optimization-ledger.md @@ -1,16 +1,43 @@ -# HermitCrab optimization ledger — tried, closed, do not retry +# HermitCrab optimization ledger — tried, closed, do not retry **Purpose: stop this work being redone.** One row per optimization attempted, what was expected, what actually happened, and the number that settled it. If you are about to try something on this -list, read its row first — several of these look irresistible on paper and three of them have +list, read its row first — several of these look irresistible on paper and three of them have already been independently rediscovered and re-retracted. Deliberately in-repo and durable, not evicted into a PR accordion. A closed avenue is only closed if the next person can find out cheaply that it is closed. -Sources: `hermitcrab-packed-forest-research.md` (theory + prior branches), -`hermitcrab-forest-memo-plan.md` (key narrowing), `hermitcrab-forest-memo-ceiling.md` -(predictions), `hermitcrab-synthesis-fold-probes.md` (measurements, sections 6–10). +**This is the only report.** The working docs it was distilled from +(`hermitcrab-packed-forest-research.md`, `hermitcrab-forest-memo-plan.md`, +`hermitcrab-forest-memo-ceiling.md`, `hermitcrab-synthesis-fold-probes.md`) are chronological +process records containing retracted intermediate claims, and are deliberately **not** merged. +They stay on the branches below, together with all instrumentation, as reproducible evidence: + +| branch | holds | +| --- | --- | +| `feature/forest-memo` | key-narrowing measurement, `RuleLengthClassifier` + 12 tests | +| `feature/synthesis-fold-probes` | the eight-bucket wall-time split, die-point histogram, sharing census, harness | +| `feature/synthesis-fold-sharing` | `SynthesisStateKey` + fold memo, parity-clean, `hits = 0` | +| `feature/per-node-cost` | `NodeCostProbe` (SIL.Machine layer) + the per-node decomposition | +| `probe/p-nogood` | row 14 — nogood subsumption census | +| `probe/p-expand` | row 16 — sound fold-entry census (trail-complete key) | +| `probe/p-battery` | rows 17, 22 — remainder split, clone-discard, Sena battery | +| `probe/p-corpus` | row 18 — corpus-scope memoization harness | +| `probe/p-contexted` | row 19 — cost-weighted die-point histogram | +| `probe/p-gen` | row 20 — generation harness | + +**No instrumentation is merged, by design.** The probes gate on `volatile bool` reads inside +`Matcher.cs` — a hot inner loop in `SIL.Machine` used by every consumer — which is an acceptable +cost for a measurement run and not one to carry in product code indefinitely. Rebuilding a probe +from a branch is cheap; maintaining research scaffolding in two product assemblies is not. + +## Net speed delivered: none + +No optimization in the "closed" section shipped, and none of them made anything faster. The +fastest thing tried was 4% **slower**. The pre-existing wins in this codebase (rows 1–3) predate +this work. What this effort produced is the measurement, the closed family, the located target, +and this ledger. --- @@ -22,39 +49,43 @@ Sources: `hermitcrab-packed-forest-research.md` (theory + prior branches), | 2 | Memoize affix-template battery (same key) | Battery was 93% of wall time | **The big win.** Reduced how *often* the battery runs | 38,840 runs -> 2,581; word 30.5 s -> **6.1 s** | | 3 | Shape sharing at clone (`CloneShareFrozenShape`) | Eager deep-copy of Shape dominates allocation | Works, allocation only | **-4.5 to -7.6% bytes**, no wall change | -## Closed — measured, does not pay +## Closed — measured, does not pay | # | Optimization | Expected | Why it failed | Number | | --- | --- | --- | --- | --- | -| 4 | **Narrow `AnalysisStateKey`** (drop strictly-shrinking rules' un-application counts) | `MaxApplicationCount` defaults to 1, so the count multiset is "which subset of rules was used" — 2^27 states. Dropping 19 of 27 rules gives 2^8 | Shape + syntactic FS + realizational FS *already* discriminate. The count component was implied by its neighbours. A worst-case bound collapsing is not the realised state count collapsing | Realised state collapse **Sena 1.12x** vs a 1.3 gate; 6 of 7 heavy words exactly **1.00**; `atawirambo` 2,556 -> 2,556 | -| 5 | **Fold-step sharing memo** (share deterministic synthesis fold steps by computed value) | Probe measured 3.22x and 8.10x shareable steps; ~50% wall ceiling on two reliable fixtures | A sound key must carry the **ordered remaining trail**. Two candidates then have to agree on their entire future to share a step, which almost never happens | **hits = 0** on the best fixture; **0.96x** (4.5% *slower* — pure key-construction cost); 1.06x on the second, inside 21.9% noise | -| 6 | **Order-insensitive synthesis-input dedupe** | 9,774x duplicate synthesis inputs | HermitCrab's morphological rules are **not order-invariant**: same rule multiset, different order, different output. Merging loses parses | 2 violations Sena, 12 independently Indonesian (e.g. `{meN, -Cont}`). Sound version: **15–40%** | -| 7 | **Trail-position indexing of synthesis rules** | 11,445,538 of 11,445,538 rejections are "this rule is not the pending trail rule" — 100% of the histogram | Each rejection is one array index plus one reference compare. Count is not cost | 11.4M x 29 ns = 0.33 s of 143 s = **0.2%** | +| 4 | **Narrow `AnalysisStateKey`** (drop strictly-shrinking rules' un-application counts) | `MaxApplicationCount` defaults to 1, so the count multiset is "which subset of rules was used" — 2^27 states. Dropping 19 of 27 rules gives 2^8 | Shape + syntactic FS + realizational FS *already* discriminate. The count component was implied by its neighbours. A worst-case bound collapsing is not the realised state count collapsing | Realised state collapse **Sena 1.12x** (state-weighted) vs a 1.3 gate; **4 of 7** heavy words exactly 1.00, `atawirambo` 2,556 -> 2,556. The three that moved -- `pidafikawo` 1.13, `cinagumanika` 1.16, **`cinacemerwa` 1.38** -- all return **zero parses**: R tracks failure, not size, and the corpus's single most expensive word *did* clear the gate. **Reopening condition:** a grammar with genuinely high R; the census is cheap and is the right first question to ask of any new grammar | +| 5 | **Fold-step sharing memo** (share deterministic synthesis fold steps by computed value) | Probe measured 3.22x and 8.10x shareable steps; ~50% wall ceiling on two reliable fixtures | A sound key must carry the **ordered remaining trail**. Two candidates then have to agree on their entire future to share a step, which almost never happens | **hits = 0** on the best fixture; **0.96x** (4.5% *slower* — pure key-construction cost); 1.06x on the second, inside 21.9% noise | +| 6 | **Order-insensitive synthesis-input dedupe** | 9,774x duplicate synthesis inputs | HermitCrab's morphological rules are **not order-invariant**: same rule multiset, different order, different output. Merging loses parses | 2 violations Sena, 12 independently Indonesian (e.g. `{meN, -Cont}`). Sound version: **15–40%** | +| 7 | **Trail-position indexing of synthesis rules** | 11,445,538 of 11,445,538 rejections are "this rule is not the pending trail rule" — 100% of the histogram | Each rejection is one array index plus one reference compare. Count is not cost | 11.4M x 29 ns = 0.33 s of 143 s = **0.2%** | | 8 | Synthesis-side length bound (Gate A) | Reject candidates that cannot reach the surface length | At the comparison point the candidate is still the bare root; its affix trail applies later inside `_synthesisRule.Apply`. Rejected valid parses | Unit suite 64 -> 34 passing. Reverted | -| 9 | Analysis length ceiling (Gate B) | Same idea, analysis side | Sound and correct. No available corpus contains the pathology it prunes | Byte-identical output, **no measurable speedup** (~4–7% slower, within noise) | +| 9 | Analysis length ceiling (Gate B) | Same idea, analysis side | Sound and correct. No available corpus contains the pathology it prunes | Byte-identical output, **no measurable speedup** (~4–7% slower, within noise) | | 10 | Lexical reachability gate (Phase 5) | Prune candidates from which no lexicon root is reachable | Both reference corpora have real compounding in their deepest stratum, which disqualifies the gate everywhere | Proven **no-op** on both corpora | | 11 | Tandem lexical intersection (T2) | Kill doomed branches early via the lexicon | The oracle only sees lexical unreachability. The expensive words fail on checks that run *after* lexical lookup succeeds | Pooled **23.5%** dead steps vs a 30% gate; one failure word at exactly **0.0%** dead | -| 12 | Pool small short-lived collections | Fewer allocations, faster | `HashSet/Dictionary.Clear()` is O(capacity); Gen0 already beats pooling at this size | -15–17% bytes but **+8.6% wall**, +12% on the parallel battery. Reverted | +| 21 | **Memo-key caching on the frozen `Word`** | **CLOSED.** Keys are built ~28x per distinct state, which looked like a cheap win. Measured cost of all `AnalysisStateKey` construction and hashing on Amharic: **3 ms of 81,707 ms = 0.0%.** Same error shape as #7: high count, no cost | +| 12 | Pool small short-lived collections | Fewer allocations, faster | `HashSet/Dictionary.Clear()` is O(capacity); Gen0 already beats pooling at this size | -15–17% bytes but **+8.6% wall**, +12% on the parallel battery. Reverted | -## Closed before building — motivation refuted by measurement +## Closed before building — motivation refuted by measurement | # | Optimization | Expected | Why it died | | --- | --- | --- | --- | -| 13 | Surface-length pruning inside the synthesis fold (P2) | "The only lever on Amharic's ~160 ms per synthesis run" | Amharic's forward synthesis is **14 ms**. The 160 ms was arithmetic on an unmeasured denominator — dividing 30 s by 186 synthesis inputs and assuming the time was there | -| 14 | Nogood lattice subsumption (P4) | Nogood hits dominate positive replays **434,628 to 25,102** — the engine mostly proves subtrees *empty*, and generalising "empty" over the feature lattice would turn nogoods from points into regions | **CLOSED on direct evidence** (it was previously closed on adjacent evidence, which is why it was reopened). Census over every empty-expansion key: `cinacemerwa` 16,176 expansions -> 16,176 distinct keys -> **16,176 singleton buckets, 0 subsumable (0.0%)**; `kukucitirani` 12,595 -> 12,595 -> **12,595 singletons, 0 subsumable**. Not one pair of nogood entries even shares a `(shape, stratum, counts, non-head)` bucket, so there is nothing for a subsumption ordering to act on. Breadth check across all 33 conformance fixtures agrees exactly: 7,493 empty expansions, 7,398 distinct keys, **0 subsumable (0.0%)** pooled, every non-empty bucket a singleton. Direction checked explicitly as `earlier.Subsumes(later)` (E subsumes Q, general over specific); the reverse would be unsound. Independently reconfirms row 4 from a different angle: the keys are maximally discriminating | -| 15 | Move constraints into the analysis key (Maxwell & Kaplan's category-splitting) | Their biggest measured win: make the chart prune what the constraint solver otherwise would | Everything synthesis rejects on is root- or realization-dependent, and the root is unknown until `LexicalLookup`. Also a Sena analysis state costs ~0.73 ms while a synthesis input costs at most ~0.12 ms — **states are dearer than synthesis inputs** | +| 13 | Surface-length pruning inside the synthesis fold (P2) | "The only lever on Amharic's ~160 ms per synthesis run" | Amharic's forward synthesis is **14 ms**. The 160 ms was arithmetic on an unmeasured denominator — dividing 30 s by 186 synthesis inputs and assuming the time was there | +| 14 | Nogood lattice subsumption (P4) | Nogood hits dominate positive replays **434,628 to 25,102** — the engine mostly proves subtrees *empty*, and generalising "empty" over the feature lattice would turn nogoods from points into regions | **CLOSED on direct evidence** (it was previously closed on adjacent evidence, which is why it was reopened). Census over every empty-expansion key: `cinacemerwa` 16,176 expansions -> 16,176 distinct keys -> **16,176 singleton buckets, 0 subsumable (0.0%)**; `kukucitirani` 12,595 -> 12,595 -> **12,595 singletons, 0 subsumable**. Not one pair of nogood entries even shares a `(shape, stratum, counts, non-head)` bucket, so there is nothing for a subsumption ordering to act on. Breadth check across all 33 conformance fixtures agrees exactly: 7,493 empty expansions, 7,398 distinct keys, **0 subsumable (0.0%)** pooled, every non-empty bucket a singleton. Direction checked explicitly as `earlier.Subsumes(later)` (E subsumes Q, general over specific); the reverse would be unsound. Independently reconfirms row 4 from a different angle: the keys are maximally discriminating | +| 15 | Move constraints into the analysis key (Maxwell & Kaplan's category-splitting) | Their biggest measured win: make the chart prune what the constraint solver otherwise would | Everything synthesis rejects on is root- or realization-dependent, and the root is unknown until `LexicalLookup`. Also a Sena analysis state costs ~0.73 ms while a synthesis input costs at most ~0.12 ms — **states are dearer than synthesis inputs** | -## Open +## Reopened this round, and their outcomes | # | Optimization | Status | | --- | --- | --- | -| 21 | **Memo-key caching on the frozen `Word`** | **CLOSED.** Keys are built ~28x per distinct state, which looked like a cheap win. Measured cost of all `AnalysisStateKey` construction and hashing on Amharic: **3 ms of 81,707 ms = 0.0%.** Same error shape as #7: high count, no cost | -| 22 | **Analysis-side rule prefilter** | 94.4% of Sena allomorph pattern matches fail (6,609,627 of 7,005,273) and 86.7% of 4,327,182 rule attempts reach matching, so the target is real | **CLOSED, cap ~10%.** The reopening hypothesis was that Sena's battery would badly understate the Amharic-derived ~11.5% bound. **It does not.** Sena `anBattery` matcher share is **21.2%**, against a >=30% gate — does not fire — giving a cap of 47.6% x 21.2% = **~10.1%**, essentially the same bound. Contended run; the gap to the gate is wide enough that contention will not close it | +| 22 | **Analysis-side rule prefilter** | 94.4% of Sena allomorph pattern matches fail (6,609,627 of 7,005,273) and 86.7% of 4,327,182 rule attempts reach matching, so the target is real | **CLOSED, cap ~10%.** The reopening hypothesis was that Sena's battery would badly understate the Amharic-derived ~11.5% bound. **It does not.** Sena `anBattery` matcher share is **21.2%**, against a >=30% gate — does not fire — giving a cap of 47.6% x 21.2% = **~10.1%**, essentially the same bound. Contended run; the gap to the gate is wide enough that contention will not close it. Caveat: the buckets are exclusive, so ~10.1% counts matcher *self*-time only — a prefilter would also avoid the failed attempts' allocations. Partially offset, though: the 21.2% includes successful matches a prefilter cannot avoid, and `AnalysisAffixTemplateRule.Apply`'s eager clone happens once per template *upstream* of any per-rule prefilter, so the prefilter cannot save it | +battery, which is only ~8% of its wall. **Sena's battery is 51.4% of its wall and has never been +decomposed per-node.** If its match share is comparable, the Sena cap is ~0.514 x 0.39 = **~20% of +wall**, outranking every other open row. Probe: `NodeCostProbe` over the 3 Sena heavy words; gate -- +matcher >=30% of Sena `anBattery` makes this the top open lead | | 16 | **`ExpandAlternatives` dedupe** | 20.7% of Sena wall, the largest bucket nobody had instrumented. Apparent census: 395,026 alternatives -> **61** distinct (0.0%), 85% of duplicates same-analysis-word | **CLOSED on the sound census.** With the pending-trail content added to the key, 395,026 -> **351,414 distinct (89.0%)** -- only **11%** are duplicates at all, and all 43,612 of them are same-analysis-word. Sound ceiling = 20.7% x 0.110 = **2.28% of Sena wall**, against a gate of >=10% to build and <5% to close. **The fourth apparent redundancy to evaporate under a complete key** | -| 17 | **Per-node cost in the analysis cascade** | 84% of the Amharic cascade is not linguistic computation; clone is 46.7% and 37.4% was unattributed | **Partly settled, remainder still open.** (a) The remainder split **MISSES its <15% gate by ~2.5x** — still ~38% on Amharic `anCascade`. Memo, Trail and **Equality each measured <0.2%**, so `Word.ValueEquals` behind the cascade's `HashSet` is *not* the missing mass, contrary to the standing hypothesis. Leading unbracketed candidate is **GC pause time**, which netstandard2.0 cannot measure directly (`GC.GetTotalPauseDuration` is .NET 7+); a collection census over 24 Amharic words gives gen0 8,217 / gen1 2,729 / gen2 301, an unusually high gen1+gen2 ratio consistent with — not proof of — GC as the remainder. (b) **Clone-discard fraction = 66.2%** (created 2,505, discarded 1,659), a count and therefore contention-immune, against a >=70% gate: **misses by ~4 points, so the build is "make a clone cheaper", not "defer cloning"**. Scoped to `AnalysisAffixTemplateRule`'s eager per-template clone, the one clone-before-verification site; the other four analysis-side sites clone only after a successful match and were excluded rather than blended in | -| 18 | Corpus-scope memoization | `AnalysisScope` dies per word, but `AnalysisStateKey` is word-independent by construction, so entries are reusable across a corpus. Nogood hits dominate positive replays 10-17x, making cross-word nogood reuse the cheapest available form of reuse | **CLOSED, and worse than a no-op.** Indonesian, 121 words, **serial on a clean machine: 1.015x, 1.4% wall-clock reduction** against a >=20% build gate and <5% close gate. Costs **68 MB** added peak managed memory. Cross-word reuse is real but small: 39 memo, 73 nogood, 237 template-nogood hits. Token-vs-type: **121 tokens, 121 distinct**, so no type-level dedupe is available either. **A first, contended run of the same harness reported 2.385x / 58.1%** with a 210.9% spread on the shared arm against 13.7% on the baseline; two testhosts were live. The counts were byte-identical across both runs — only the timing was contaminated, and min-of-N on a high-spread arm picks the luckiest sample. Analysis-set equality held throughout, so the key genuinely is word-independent. **Sena, first 300 words, is the decisive case: the shared scope is 28-32% *slower* than per-word scopes** (two clean serial runs, no contention: 282.7s vs 372.2s and 308.9s vs 396.7s baseline-vs-shared; deterministic counters byte-identical across both runs), and costs **~29.3 GB** added peak managed memory against a 2 GB budget. Root cause: `AnalysisScope.MaxMemoEntries` (100,000) is a *per-scope*, not per-word, budget. Two heavy early words (#74, #76 of 300) alone fill the shared `Memo` table to the cap and `TemplateMemo` to 55,071; every word after that loses the free within-word memoization a fresh per-word scope would have given it for nothing, because `Store` silently no-ops past the cap (see its comment) with no eviction. Cross-word hits (shared minus baseline) are **negative** — memo -77,648, nogood -747,607 — i.e. sharing produced *fewer* total hits than 300 independent fresh-table runs would have on their own; the small `+820` template-hit / `+9` template-nogood gain does not come close to covering it. Amharic, capped at 28 words (word 29 hangs, excluded per standing guidance): **0.9% wall-clock reduction**, table nowhere near the cap (Memo 1,251, TemplateMemo 652 of 100,000), consistent with row 17's finding that Amharic's cost is dominated by clone/allocation per node rather than distinct-state count, so avoiding a handful of cross-word re-derivations barely moves wall time. Token-vs-type on all three corpora, at the runs' own sizes and on each corpus's full word list: **zero duplicate tokens** — whole-word caching is not available as a cheaper alternative anywhere here. No analysis-set divergence on any corpus, at any scope regime: the stop condition that would have put rows 1-3 in question did not fire. **A shared-but-capped memo table is not "the same optimization, smaller ceiling" — past the cap it actively takes away the free per-word memoization the product already has, and retains every live entry for the run's full duration instead of per-word, which is where the 29 GB comes from.** Any future attempt needs real eviction (LRU, generational, or per-word sub-budgets) before this is worth re-measuring, and even then the >=20% gate looks unreachable given Indonesian's ceiling under an uncapped-in-practice regime was already only 1.4% | -| 19 | Contexted constraints / abstract feature-only replay | **CLOSED.** Cost-weighted the P1b die-point histogram (count → wall-time per die point, gated before the run: shape-free cost ≥60% builds, <30% closes). Sena `atawirambo` **6.0%**, `cinacemerwa` **0.0%**, Amharic 28 words **0.4%** — all fire the close gate, none come close to the middle band. All 33 conformance fixtures measured too (breadth, not part of the gate): 24 close, 3 fire the build gate (`morphotactic-attribute-breadth` 67.1%, `mpr-group-overwrite-without-realizational` 91.5%, `prefixal-discontinuous-slot-dependency` 71.8%), 3 land in the undecided 30–60% band, 2 have zero rejections, 1 has zero words. **All three build-gate fixtures are sub-50 ms** (38.15 ms / 1.49 ms / 4.28 ms wall) — the method-rules-earned "cannot carry a timing claim" flag, so they do not override the real-corpus verdict. On the three real grammars, count and cost agree in direction (`RuleNotApplicableOrPatternMismatch`, shape-dependent, dominates both). Among the fixtures, one reliable one (wall ≥50 ms) shows the count/cost divergence the gate exists to catch: `suffixing-extension-slot-ordering` (53 words, 63.98 ms) shape-free count 12.9% vs. cost 53.3% — cost roughly 4x count, same direction as the ApplicationCount/MprFeatures/FeatureUnification checks costing more per event than pattern-mismatch's near-O(1) trail check. (`feature-gating-breadth`, 28.6% vs. 56.8%, shows the same direction but is sub-50 ms and unreliable, same flag as the three build-gate fixtures.) Orthogonal to classification: die-point *rejection* cost itself is negligible against wall time on all three real-corpus runs regardless of class (atawirambo ~19.8 ms of 11,547 ms wall = 0.17%; cinacemerwa ~112.9 ms of 57,275 ms = 0.20%; Amharic ~88.8 ms of 91,456 ms = 0.10%) — reconfirming row 7/6.2's finding that rejections are cheap. Abstract replay would be optimizing a phase that costs approximately nothing on the two grammars this project treats as ground truth. Not built | -| 20 | **Row 5's family on `GenerateWords`** (generation, ~100% synthesis share vs parsing's ~5%/~0.3%) | Row 20 was unmeasured; generation's share is ~100% so row 5's fold ratios would apply at face value if they transferred. `UseSynthesisFoldMemo` had never been wired into `GenerateWords` (only into the parse-then-synthesize path) — wired in under the same untraced/`MaxDegreeOfParallelism==1` restriction to make this measurable at all | Hits are no longer uniformly zero: 3 of 23 lookup-reaching fixtures clear the 5% gate (diacritic-segments 33.3%, suffixing-vowel-harmony 28.6%, suffixing-evidential-adjacency-chain 37.7%; a 4th sits just under it at 4.8%), reopening row 5 narrowly for generation. But the reopened case recloses immediately: the A/B on the best fixture shows the wall-clock "win" sitting inside its own noise floor, and the pooled hit rate is dominated by `deep-optional-affix-nesting` (925 of 1,613 lookups) at hits=0, reproducing row 5's exact 4.5%-slower key-construction cost on the identical fixture | Parity: 0 divergences, 1,236 generations, 32/33 fixtures (33rd throws identically both sides). Best case: 130 hits / 5 reps (37.7% hit rate), **7.8%** wall-clock reduction inside a **22.2%** off-arm spread — not a result. 19/23 lookup-reaching fixtures at exactly **0%**; pooled hit rate **2.67%** (43/1,613), skewed by the one large zero-hit fixture | +| 17 | **Per-node cost in the analysis cascade** | 84% of the Amharic cascade is not linguistic computation; clone is 46.7% and 37.4% was unattributed | **Partly settled, remainder still open.** (a) The remainder split **MISSES its <15% gate by ~2.5x** — still ~38% on Amharic `anCascade`. Memo, Trail and **Equality each measured <0.2%**, so `Word.ValueEquals` behind the cascade's `HashSet` is *not* the missing mass, contrary to the standing hypothesis. **Two caveats the first write-up missed.** (i) The ~38% figure comes from a *contended* run — three timing probes were live — and descheduled wall time lands preferentially in the remainder while deflating every self-time share, so **CPU starvation is itself an unbracketed candidate biasing in exactly the direction observed**. (ii) The claim that GC pause time is unmeasurable is **wrong at the measurement point**: only the library assemblies are netstandard2.0; the probe harness lives in `tests/SIL.Machine.Morphology.HermitCrab.Tests`, which targets **net10.0**, so `GC.GetTotalPauseDuration()` and `GC.GetGCMemoryInfo().PauseTimePercentage` are directly callable. Next probe, gated: re-run serial — remainder <20% means contention was the missing mass and 17(a) recloses on the cheaper-clone target; still >=30% then bracket with `GetTotalPauseDuration()` deltas — ΔPause >=50% of the remainder's absolute ms confirms GC (build target converges with clone/freeze at 46.7%), <=15% refutes it and the next split is enumerator/LINQ orchestration (the 2026-07 alloc audit already flagged `ExecuteCommands` enumerator boxing); a collection census over 24 Amharic words gives gen0 8,217 / gen1 2,729 / gen2 301, an unusually high gen1+gen2 ratio consistent with — not proof of — GC as the remainder. (b) **Clone-discard fraction = 66.2%** (created 2,505, discarded 1,659), a count and therefore contention-immune, against a >=70% gate: **misses by ~4 points, so the build is "make a clone cheaper", not "defer cloning"**. Scoped to `AnalysisAffixTemplateRule`'s eager per-template clone, the one clone-before-verification site; the other four analysis-side sites clone only after a successful match and were excluded rather than blended in | +| 18 | Corpus-scope memoization | `AnalysisScope` dies per word, but `AnalysisStateKey` is word-independent by construction, so entries are reusable across a corpus. Nogood hits dominate positive replays 10-17x, making cross-word nogood reuse the cheapest available form of reuse | **CLOSED, and worse than a no-op.** Indonesian, 121 words, **serial on a clean machine: 1.015x, 1.4% wall-clock reduction** against a >=20% build gate and <5% close gate. Costs **68 MB** added peak managed memory. Cross-word reuse is real but small: 39 memo, 73 nogood, 237 template-nogood hits. Token-vs-type: **121 tokens, 121 distinct**, so no type-level dedupe is available either. **A first, contended run of the same harness reported 2.385x / 58.1%** with a 210.9% spread on the shared arm against 13.7% on the baseline; two testhosts were live. The counts were byte-identical across both runs — only the timing was contaminated, and min-of-N on a high-spread arm picks the luckiest sample. Analysis-set equality held throughout, so the key genuinely is word-independent. **Sena, first 300 words, is the decisive case: the shared scope is 28-32% *slower* than per-word scopes** (two clean serial runs, no contention: 282.7s vs 372.2s and 308.9s vs 396.7s baseline-vs-shared; deterministic counters byte-identical across both runs), and costs **~29.3 GB** added peak managed memory against a 2 GB budget. Root cause: `AnalysisScope.MaxMemoEntries` (100,000) is a *per-scope*, not per-word, budget. Two heavy early words (#74, #76 of 300) alone fill the shared `Memo` table to the cap and `TemplateMemo` to 55,071; every word after that loses the free within-word memoization a fresh per-word scope would have given it for nothing, because `Store` silently no-ops past the cap (see its comment) with no eviction. Cross-word hits (shared minus baseline) are **negative** — memo -77,648, nogood -747,607 — i.e. sharing produced *fewer* total hits than 300 independent fresh-table runs would have on their own; the small `+820` template-hit / `+9` template-nogood gain does not come close to covering it. Amharic, capped at 28 words (word 29 hangs, excluded per standing guidance): **0.9% wall-clock reduction**, table nowhere near the cap (Memo 1,251, TemplateMemo 652 of 100,000), consistent with row 17's finding that Amharic's cost is dominated by clone/allocation per node rather than distinct-state count, so avoiding a handful of cross-word re-derivations barely moves wall time. Token-vs-type on all three corpora, at the runs' own sizes and on each corpus's full word list: **zero duplicate tokens** — whole-word caching is not available as a cheaper alternative anywhere here. No analysis-set divergence on any corpus, at any scope regime: the stop condition that would have put rows 1-3 in question did not fire. **A shared-but-capped memo table is not "the same optimization, smaller ceiling" — past the cap it actively takes away the free per-word memoization the product already has, and retains every live entry for the run's full duration instead of per-word, which is where the 29 GB comes from.** Any future attempt needs real eviction (LRU, generational, or per-word sub-budgets) before this is worth re-measuring, and even then the >=20% gate looks unreachable given Indonesian's ceiling under an uncapped-in-practice regime was already only 1.4% | +| 19 | Contexted constraints / abstract feature-only replay | **CLOSED.** Cost-weighted the P1b die-point histogram (count → wall-time per die point, gated before the run: shape-free cost ≥60% builds, <30% closes). Sena `atawirambo` **6.0%**, `cinacemerwa` **0.0%**, Amharic 28 words **0.4%** — all fire the close gate, none come close to the middle band. All 33 conformance fixtures measured too (breadth, not part of the gate): 24 close, 3 fire the build gate (`morphotactic-attribute-breadth` 67.1%, `mpr-group-overwrite-without-realizational` 91.5%, `prefixal-discontinuous-slot-dependency` 71.8%), 3 land in the undecided 30–60% band, 2 have zero rejections, 1 has zero words. **All three build-gate fixtures are sub-50 ms** (38.15 ms / 1.49 ms / 4.28 ms wall) — the method-rules-earned "cannot carry a timing claim" flag, so they do not override the real-corpus verdict. On the three real grammars, count and cost agree in direction (`RuleNotApplicableOrPatternMismatch`, shape-dependent, dominates both). Among the fixtures, one reliable one (wall ≥50 ms) shows the count/cost divergence the gate exists to catch: `suffixing-extension-slot-ordering` (53 words, 63.98 ms) shape-free count 12.9% vs. cost 53.3% — cost roughly 4x count, same direction as the ApplicationCount/MprFeatures/FeatureUnification checks costing more per event than pattern-mismatch's near-O(1) trail check. (`feature-gating-breadth`, 28.6% vs. 56.8%, shows the same direction but is sub-50 ms and unreliable, same flag as the three build-gate fixtures.) Orthogonal to classification: die-point *rejection* cost itself is negligible against wall time on all three real-corpus runs regardless of class (atawirambo ~19.8 ms of 11,547 ms wall = 0.17%; cinacemerwa ~112.9 ms of 57,275 ms = 0.20%; Amharic ~88.8 ms of 91,456 ms = 0.10%) — reconfirming row 7/6.2's finding that rejections are cheap. Abstract replay would be optimizing a phase that costs approximately nothing on the two grammars this project treats as ground truth. Not built | +| 20 | **Row 5's family on `GenerateWords`** (generation, ~100% synthesis share vs parsing's ~5%/~0.3%) | Row 20 was unmeasured; generation's share is ~100% so row 5's fold ratios would apply at face value if they transferred. `UseSynthesisFoldMemo` had never been wired into `GenerateWords` (only into the parse-then-synthesize path) — wired in under the same untraced/`MaxDegreeOfParallelism==1` restriction to make this measurable at all | Hits are no longer uniformly zero: 3 of 23 lookup-reaching fixtures clear the 5% gate (diacritic-segments 33.3%, suffixing-vowel-harmony 28.6%, suffixing-evidential-adjacency-chain 37.7%; a 4th sits just under it at 4.8%), reopening row 5 narrowly for generation. But the reopened case recloses immediately: the A/B on the best fixture shows the wall-clock "win" sitting inside its own noise floor, and the pooled hit rate is dominated by `deep-optional-affix-nesting` (925 of 1,613 lookups) at hits=0, reproducing row 5's exact 4.5%-slower key-construction cost on the identical fixture | Parity: 0 divergences, 1,236 generations, 32/33 fixtures (33rd throws identically both sides). Best case: 130 hits / 5 reps (37.7% hit rate), **7.8%** wall-clock reduction inside a **22.2%** off-arm spread — not a result. 19/23 lookup-reaching fixtures at exactly **0%**; pooled hit rate **2.67%** (43/1,613), skewed by the one large zero-hit fixture | --- @@ -64,7 +95,7 @@ Three independent measurements, three boundaries, same collapse: | measurement | key used | apparent | sound | | --- | --- | --- | --- | -| synthesis-input dedupe (#6) | order-insensitive | 9,774x | 15–40% | +| synthesis-input dedupe (#6) | order-insensitive | 9,774x | 15–40% | | fold-entry census | trail position only | 6,476x | not established | | fold-step sharing (#5) | trail position only | 3.22x / 8.10x | **hits = 0** | @@ -72,22 +103,27 @@ Three independent measurements, three boundaries, same collapse: step distinct, and every measurement showing large shareable work is measuring a key that omits it.** The thing that makes sharing visible is the thing that makes sharing wrong. -That closes a *family*: packed parse forests, fold-step sharing, and synthesis-input dedupe all -require distinct derivations to converge on a genuinely identical state. In this engine they do -not converge. It is the same fact as the rules being non-order-invariant, seen from the other side. +That closes a *family*: packed parse forests **of the merge-by-key kind**, fold-step sharing and +synthesis-input dedupe all require distinct derivations to converge on a genuinely identical +state. In this engine they do not converge -- the same fact as the rules being non-order-invariant, +seen from the other side. **It does not close row 19**: a contexted-constraint forest packs +*without* merging states, so this argument does not reach it. Row 19 stays open and unmeasured. ## Where the time actually is -Measured with eight exclusive buckets summing to wall (`unaccounted` 0.1% on Amharic, 1.5% on Sena): +Measured with eight exclusive buckets summing to wall. Amharic is a single run (`unaccounted` +0.1%). **Sena is spliced from two runs -- see the note in its row.** | grammar | breakdown | | --- | --- | -| **Sena** heavy words | analysis template battery **51.4%** (still, after the 5x memo), `ExpandAlternatives` **20.3%**, analysis cascade 18.4%, **all synthesis ~5%** | +| **Sena** heavy words | battery **51.4%**, cascade 18.4%, all synthesis ~5% -- *pre-`ExpandAlternatives` run (143,303 ms, unaccounted 20.1%)*; `ExpandAlternatives` **20.3%**, unaccounted 1.5% -- *later run (112,713 ms)*. **Two runs: not co-summable.** One re-run would put all Sena shares on one denominator | | **Amharic** | analysis cascade **~95%** of analysis, which is 99.5% of wall; **all synthesis 0.3%** | **Every optimization in this ledger that worked reduced how many nodes are visited. None touched -what a node costs.** Amharic spends **~170 ms per analysis state** in a cascade already at its -state-count floor. +what a node costs.** Amharic spends **~170 ms per analysis state**. (The *measured* state-count floor -- 2,555 +expansions against 2,546 states -- is Sena `atawirambo`. For Amharic, near-floor behaviour is +**inferred** from R ~ 1, which compares the full key against the narrowed key, not against a +semantic floor.) ### Inside one node (Amharic, 28 words, both probe layers live) @@ -99,30 +135,61 @@ state-count floor. | FeatureStruct algebra | 3.4% | 3.5% | | memo-key construction | 0.0% | 0.0% | -Censuses: 27,385 rule attempts — guards reject 19.5% (application-count 6.3%, unifiability 13.2%), +Censuses: 27,385 rule attempts — guards reject 19.5% (application-count 6.3%, unifiability 13.2%), **80.5% reach matching**. Of 25,178 allomorph pattern matches, **1,938 succeed (7.7%)**, 23,240 fail (**92.3%**). -**The headline: only ~16% of the Amharic analysis cascade is linguistic computation.** Pattern -matching plus feature unification is 15.9%. Clone plus unattributed machinery is **84%**. The -engine spends its time constructing and discarding objects, not deciding linguistic questions. +**The headline: only ~16% of the Amharic analysis cascade is *identified* as linguistic +computation** -- matching 12.5% plus feature unification 3.4%. Clone/freeze is **46.7%** and +**37.4% is unattributed**, which is unknown rather than known-non-linguistic. The defensible +statement is that at most a sixth of the cascade has been shown to decide linguistic questions; +of the rest, half is measured (clone) and half is still open. -**Gate outcomes, written before the run.** "Any bucket >=50% becomes the sole build target" — clone +**Gate outcomes, written before the run.** "Any bucket >=50% becomes the sole build target" — clone is 46.7%, does not fire. "Pattern matching dominates *and* >=80% of attempts fail -> build a -prefilter" — 92.3% do fail, but matching is 12.5%, so it does not dominate; does not fire. -**"Remainder >=30% means the decomposition is incomplete and that is the result" — 37.4%, FIRES.** +prefilter" — 92.3% do fail, but matching is 12.5%, so it does not dominate; does not fire. +**"Remainder >=30% means the decomposition is incomplete and that is the result" — 37.4%, FIRES.** So there is no declared build target yet. The next probe splits the 37.4%: `RuleCascade` / enumerator orchestration, trail dictionary operations, LINQ allocation, memo lookups, GC. **A caution on the 46.7% before anyone acts on it.** `Word.Clone` being half the cascade does not -by itself mean cloning is waste — the cascade forks a `Word` per rule application precisely so it +by itself mean cloning is waste — the cascade forks a `Word` per rule application precisely so it can explore alternatives without mutating shared state. The deciding question is what fraction of clones are *discarded unused*: cloned, rejected by a guard or a failed match, thrown away. Given -92.3% of pattern matches fail, that fraction is plausibly very high, but it is unmeasured. If most +92.3% of pattern matches fail, that fraction has now been **measured at 66.2%** (row 17b), which misses the >=70% gate and points at the cheaper-clone build rather than the defer-clone one. If most clones are discarded, the lever is **not cloning until a rule commits**. If most survive, the lever is **making a clone cheaper** (copy-on-write shape, which the RUSTIFY branch already explored). -Different builds; measure first. +Different builds; measure first. **Gate: probe the clone-discard fraction; >=70% discarded means +the defer-clone build, below that the cheaper-clone build.** + +## Two durable code constraints + +Neither is expressible in code, and both bite silently. + +1. **`Word.ReplayOnto` does not splice `_mrulesUnapplied`.** It splices `_mruleApps` and + `_nonHeadApps` only. This is safe *solely* because `AnalysisStateKey` includes the per-rule + count multiset, which guarantees a memo hit has identical arrival and stored-arrival counts. + **Anyone narrowing that key breaks correctness with no test failure.** Fix, if the key ever + changes: store arrival counts on `MemoEntry` and compute `stored - storedArrival + query` — a + no-op under today's key, which is also how to test it. This is the one code change from this + work worth making on its own merits, as defensive hardening. +2. **HermitCrab's morphological rules are not order-invariant.** Same pending-rule multiset, + different application order, different synthesis output — 2 cases on Sena, 12 independently on + Indonesian (e.g. `{meN, -Cont}`). This bounds every packed-readout scheme. It is the assumption + Maxwell and Kaplan (CL 19(4):571–590) rely on and this engine does not satisfy. + +## Calibration record — predictions vs outcomes + +Written before the measurements, scored after. Kept because the misses are informative. + +| prediction | outcome | +| --- | --- | +| Sena key collapse 2–4x | **1.12x** — wrong, and it was the number that decided the project | +| Indonesian key collapse 1.0–1.3x | 1.17x — right | +| Amharic "most likely to disappoint" | best static classification of the three (31 of 36 rules) — wrong | +| Fold sharing ~50% wall ceiling | **0.96x realised** — wrong; the ceiling was measured with a key that would lose parses | +| Amharic synthesis-bound at ~160 ms/run | forward synthesis is **14 ms** — wrong; arithmetic on an unmeasured denominator | ## Method rules earned here @@ -139,5 +206,27 @@ Different builds; measure first. 5. **Three grammars minimum, reported unpooled.** Maxwell & Kaplan measured a 100x swing between two variants of one grammar. A pooled average across fixtures of different size has had to be retracted twice in this project. -6. **Search completeness is never traded for speed.** HermitCrab is the permanent fallback engine +6. **Min-of-N is only valid when the arm's spread is small relative to the effect.** Row 18's + first run read 2.385x with a 210.9% spread on one arm and 13.7% on the other; serial and clean + it is **1.015x**. Min-of-N picks each arm's luckiest sample independently, so a high-spread arm + wins spuriously. Print the spread beside every speedup and refuse a number whose spread exceeds + it. +7. **Never dispatch timing-sensitive probes concurrently.** Three at once put four testhosts on the + machine and invalidated the bucket percentages in rows 17 and 22 as well as row 18's headline. + Counts survive contention; shares and speedups do not. +8. **A shared bounded cache without eviction is not a smaller version of the optimization** — it + competes with itself for a fixed budget and can land strictly worse than no sharing. Row 18's + shared scope produced *negative* cross-word hits. +9. **Never aggregate the test logger's console output.** It prints every line twice, the second + copy space-prefixed; a `^\s*` grep doubles every total. This produced a 2x error in row 20's + first write-up. +10. **Search completeness is never traded for speed.** HermitCrab is the permanent fallback engine behind the FST work. +7. **A bucket reporting exactly `0.00` is broken until proven otherwise.** `NodeCostProbe.Enabled` + was declared and read in `Matcher.cs` but set by nothing — two gates in two assemblies, one + wired. Both lower-layer buckets read `0.00ms`, which looks like "measured and negligible" + rather than "never ran", and their time silently inflated the remainder. Wire cross-assembly + probe gates from a single place. +8. **Always carry an explicit remainder column.** It caught the wall-split misattribution and the + dead probe gate. Four of the five retractions in this work would have been caught at the point + of measurement by a remainder that did not sum. From 004e1990fbcc73b297bb8e797cccc1189f63b45f Mon Sep 17 00:00:00 2001 From: John Lambert Date: Thu, 27 Aug 2026 16:07:30 -0400 Subject: [PATCH 5/5] docs: add the probe design record; ledger no longer depends on the branches The seven probe branches are being deleted, which would leave the ledger's evidence table dangling. hermitcrab-probe-design.md replaces it. It carries: the two-assembly probe architecture and the gate trap that makes a bucket read exactly 0.00ms; the eight exclusive wall-time slices and why synForward must be net of cascade+battery (the 25x error); the per-node decomposition and how it avoids double-counting across assemblies; the census designs and why die points must be cost-weighted (100% of events, 0.2% of time); all three key designs with field-level audits, including why the ordered remaining trail is load-bearing and why Word.ValueEquals is not a starting point; the RuleLengthClassifier rules and the un-application direction trap; the full A/B methodology with the three failures that forced each part; and the shared-scope concurrency hazard. Restates the two durable engine facts standalone so they survive independently of any row: ReplayOnto not splicing _mrulesUnapplied, and MaxMemoEntries being a per-scope cap with no eviction. Co-Authored-By: Claude Opus 5 --- docs/hermitcrab-optimization-ledger.md | 25 ++- docs/hermitcrab-probe-design.md | 226 +++++++++++++++++++++++++ 2 files changed, 237 insertions(+), 14 deletions(-) create mode 100644 docs/hermitcrab-probe-design.md diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md index 27ec0bb2..f4c8228d 100644 --- a/docs/hermitcrab-optimization-ledger.md +++ b/docs/hermitcrab-optimization-ledger.md @@ -12,20 +12,17 @@ if the next person can find out cheaply that it is closed. (`hermitcrab-packed-forest-research.md`, `hermitcrab-forest-memo-plan.md`, `hermitcrab-forest-memo-ceiling.md`, `hermitcrab-synthesis-fold-probes.md`) are chronological process records containing retracted intermediate claims, and are deliberately **not** merged. -They stay on the branches below, together with all instrumentation, as reproducible evidence: - -| branch | holds | -| --- | --- | -| `feature/forest-memo` | key-narrowing measurement, `RuleLengthClassifier` + 12 tests | -| `feature/synthesis-fold-probes` | the eight-bucket wall-time split, die-point histogram, sharing census, harness | -| `feature/synthesis-fold-sharing` | `SynthesisStateKey` + fold memo, parity-clean, `hits = 0` | -| `feature/per-node-cost` | `NodeCostProbe` (SIL.Machine layer) + the per-node decomposition | -| `probe/p-nogood` | row 14 — nogood subsumption census | -| `probe/p-expand` | row 16 — sound fold-entry census (trail-complete key) | -| `probe/p-battery` | rows 17, 22 — remainder split, clone-discard, Sena battery | -| `probe/p-corpus` | row 18 — corpus-scope memoization harness | -| `probe/p-contexted` | row 19 — cost-weighted die-point histogram | -| `probe/p-gen` | row 20 — generation harness | +They are preserved only as this record. **The evidence branches were deleted after the round +closed**, along with all instrumentation: research scaffolding is not worth maintaining in two +product assemblies, since the probes gate on `volatile bool` reads inside `Matcher.cs`, a hot inner +loop in `SIL.Machine` used by every consumer. + +**`docs/hermitcrab-probe-design.md` is the replacement.** It carries the probe architecture and its +two-assembly gate trap, the eight exclusive wall-time buckets and why `synForward` must be recorded +net of cascade and battery, the per-node decomposition, the census designs, all three key designs +with field-level audits, the classifier's direction trap, and the full A/B methodology with the +three measurement failures that forced each part of it — enough to rebuild any probe in a day. The +expensive part was never the code; it was learning what to measure and which measurements lie. **No instrumentation is merged, by design.** The probes gate on `volatile bool` reads inside `Matcher.cs` — a hot inner loop in `SIL.Machine` used by every consumer — which is an acceptable diff --git a/docs/hermitcrab-probe-design.md b/docs/hermitcrab-probe-design.md new file mode 100644 index 00000000..b1bb04ea --- /dev/null +++ b/docs/hermitcrab-probe-design.md @@ -0,0 +1,226 @@ +# HermitCrab probe design — enough to rebuild it + +Companion to `hermitcrab-optimization-ledger.md`. That document records *what was tried and why it +failed*. This one records *how it was measured*, in enough detail to reconstruct the instrumentation +and the keys from scratch. + +**Why this exists:** the seven probe branches that produced the ledger's numbers were deleted after +the round closed. Research scaffolding in two product assemblies is not worth maintaining — the +probes gate on `volatile bool` reads inside `Matcher.cs`, a hot inner loop in `SIL.Machine` used by +every consumer. So the code is gone and this is the design record. Anything here can be rebuilt in a +day; the expensive part was learning what to measure and which measurements lie. + +--- + +## 1. Instrumentation architecture + +Two probe classes, in two assemblies, with **two independent gates**. + +`SynthesisProbe` in `SIL.Machine.Morphology.HermitCrab`, and `NodeCostProbe` in `SIL.Machine`. Each +has its own `internal static volatile bool Enabled`, default `false`. **Both must be set.** Setting +only the HermitCrab one makes every lower-layer bucket read exactly `0.00ms`, which looks like +"measured and negligible" rather than "never ran" — this cost a round (ledger method rule: a bucket +reporting exactly zero is broken until proven otherwise). Wire both from one place in the harness. + +Everything is insertion-only: a `RecordDie(...)` or `Add*Ticks(...)` call placed immediately before +an existing `return` / `continue`, never a change to control flow. Where a site read +`else if (IsTracing) { trace }`, it becomes `else { if (IsTracing) { trace } Record(...) }` — +behaviour-preserving. Counters are `Interlocked`; timers are `Stopwatch.GetTimestamp()` deltas +accumulated with `Interlocked.Add`, with `Reset*` methods so a harness reads per-word deltas. + +### The wall-time split + +**Eight mutually exclusive top-level slices that sum to wall**, measured around +`Morpher.ParseWord`: + +`lookup` (brackets `LexicalLookup`, materialised with `.ToList()` so the bracket covers only the +lookup itself) · `synCascade` · `synBattery` · `synForward` · `synExpand` (brackets +`Word.ExpandAlternatives()` in `SynthesizeSequential`) · `anTotal` (brackets the whole +`_analysisRule.Apply(input)` call) · `unaccounted` (defined as the remainder, never tracked). + +`synForward` is recorded **net of** `synCascade` and `synBattery`: capture both counters before the +`_synthesisRule.Apply` call and subtract their deltas after. This matters enormously and caused a +25x published error — the work a synthesis fold memo would share runs *inside* the cascade and +battery brackets, so the share to divide by is `synCascade + synBattery + synForward`, not +`synForward` alone. Template slot rules compile to the same `SynthesisAffixProcessRule` classes via +`RuleBatch` (`SynthesisAffixTemplateRule` ctor), which is why they land in `synBattery`. + +Nested *inside* `anTotal`, as a breakdown rather than additional slices: `anCascade` (the +`_mrulesRule.Apply` call in `AnalysisStratumRule.ApplyMorphologicalRules`), `anBattery` +(`ApplyTemplateBattery`), `anPhono` (`_prulesRule.Apply`). + +**Always print the remainder as its own column.** It caught the misattributed split, the dead probe +gate, and would have caught four of five retractions at the point of measurement. + +### The per-node decomposition + +Inside `anCascade` and `anBattery`, sub-buckets: `match` · `feature` · `clone` · `key` · `memo` · +`trail` · `equality` · explicit remainder. + +`match` and `feature` come from the `SIL.Machine` layer. To avoid double counting, the HermitCrab +side reads `NodeCostProbe.MatcherTicks` / `FeatureTicks` **before and after** the cascade call and +records the delta — so the lower-layer buckets are slices *inside* the HermitCrab bucket, not +alongside it. Matcher self-time has nested work subtracted at the source (inside `Matcher.cs`); +feature work needs a re-entrancy depth guard or recursive `Unify` calls double-count. + +Measured outcome, so a rebuild can sanity-check itself: on Amharic `anCascade`, clone **46.7%**, +remainder **37.4%**, match 12.5%, feature 3.4%, key **0.0%**, and memo / trail / equality each +**<0.2%**. + +### Censuses + +Rule attempts, split by which guard rejected them (`guardSelector`, +`guardApplicationCount`, `guardUnifiability`, `reachedMatching`); allomorph pattern-match +attempts vs matches; clone created vs discarded vs kept. + +Die points, as an enum with a counter array **and a parallel tick array** — timing each individual +guard check, not the enclosing `Apply()`. Count and cost diverge: `RuleNotApplicableOrPatternMismatch` +was **100.0% of events** and about **0.2% of time**, because each rejection is one list index plus one +reference compare (~29 ns). Never report a die-point histogram by count alone. + +Die points, classified for the abstract-replay question — **shape-free**: synthesis-side +`MaxApplicationCount`, syntactic-feature unification, MPR features, realizational +subsumption/blocking, co-occurrence and obligatory features in `IsWordValid`. **Shape-dependent**: +allomorph pattern match / rule-not-applicable, allomorph environments, final `IsMatch`. +`LexicalLookupMiss` is neither — it precedes any synthesis step. + +--- + +## 2. The keys + +### `AnalysisStateKey` (pre-existing) + +`(Shape, Stratum, SyntacticFeatureStruct, RealizationalFeatureStruct, NonHeadCount, per-rule +un-application count multiset)`. Order-independent by design — no analysis-side rule reads trail +order, which is the redundancy it collapses. The multiset is XOR-combined, not `*31`-rolled, so it +hashes commutatively. + +**Freeze-on-read in the constructor is mandatory.** `AnalysisAffixTemplateRule.Apply` reassigns +`SyntacticFeatureStruct` to a fresh unfrozen clone *after* the owning `Word` is frozen, and that +setter has no `CheckFrozen()` guard. + +### The narrowed variant (row 4) + +Filter the count multiset to rules that are **not** provably shrinking. Two requirements learned the +hard way: build the retain-map **once per grammar**, not per key (key construction is on the hottest +path); and filter during hashing and equality rather than by materialising a filtered dictionary, so +the narrow key allocates no more than the full one. **Absent from the map means retain** — a rule the +classifier walk does not cover must be counted, never silently dropped. + +A `retainedCount` field is needed so equality can compare sizes without enumerating. + +### `SynthesisStateKey` (row 5) — and the trap + +Everything a synthesis step reads: shape+annotations, syntactic FS, realizational FS, MPR set, root +allomorph, disjunctive allomorph indices, applied-rule counts, `IsPartial`, +`IsLastAppliedRuleFinal`, stratum, **and the ordered remaining trail** — +`Word.MorphologicalRuleTrail` sliced to `[0, PendingTrailPosition]`. + +**The trail content is the whole game.** A key with `PendingTrailPosition` (an integer index) and no +trail content is fine for *measuring* and catastrophic for *skipping work*: two candidates at the +same index with different pending sequences compare equal, and merging them discards one +derivation's continuation. Every large apparent redundancy in this engine was measured with such a +key, and each collapsed when the content was added — 9,774x → 15–40%, 61-of-395,026 → 351,414-of-395,026, +3.22x/8.10x → `hits = 0`. + +**`Word.ValueEquals` is not a starting point.** It omits `_syntacticFS`, MPR features and the +disjunctive allomorph indices, all of which synthesis reads. Inheriting from it produces a silently +inflated ratio. + +Two more properties a real memo needs: stored results are **sets**, not values (realizational rules +are trail-exempt — `SynthesisRealizationalAffixProcessRule` has no `IsMorphologicalRuleApplicable` +gate — and several allomorphs of one rule can legitimately match one input before the disjunctive +break); and stored outputs embed the producing candidate's trail, so replay must **re-anchor**, the +mirror of `Word.ReplayOnto`. + +### `RuleLengthClassifier` (row 4's static half) + +Verdicts: `Shrinking` / `NonShrinking` / `Unknown`, where `Unknown` is treated identically to +`NonShrinking` by the key and kept distinct only so a census can tell "this grammar has zero +morphemes" from "this grammar has constructs we cannot analyse". + +An `AffixProcessAllomorph` shrinks iff **every** allomorph (a) inserts at least one *segment* +(`InsertSegments` counting only `HCFeatureSystem.Segment` nodes — a boundary-only insertion is +conservatively NonShrinking, since `InsertSegments.GenerateAnalysisLhs` omits boundaries from the +analysis pattern; `InsertSimpleContext` counts as 1), (b) has **every Lhs part captured** by a Rhs +`CopyFromInput` or `ModifyFromInput`, and (c) copies no part twice. + +The direction trap, which is easy to invert: on the analysis side rules are *un*-applied, so a rule +that inserts on synthesis **removes** on analysis, and a rule that truncates on synthesis +**untruncates** — an Lhs part with no Rhs copy makes the un-applied word **grow**. The authority is +`AnalysisMorphologicalTransform.GenerateShape`, which walks the Lhs parts and either copies the +captured span or calls `Untruncate` to regenerate the part from its pattern. Reduplication (a part +copied twice) and `CompoundingRule` are `Unknown`; conservatism only costs pruning opportunity. + +Measured census, for a rebuild to check against: Sena 19 of 27 rules shrinking, Indonesian 10 of 15, +Amharic 31 of 36. + +--- + +## 3. Harness design + +`[TestFixture]`, `[Explicit]`, environment-driven, writing only `TestContext` lines and no files — +this repo never commits real grammars, and derived corpus data must not leak into a committed path. +**Never print rule, stratum, morpheme or lexical-entry names, or word lists.** Counts, ratios and +timings only. Conformance fixture IDs are committed and safe. + +Env: `HC_MEMO_GRAMMAR`, `HC_MEMO_WORDS`, `HC_MEMO_MAX_WORDS`, `HC_PROBE_WORDS` (explicit +comma-separated list), `HC_PROBE_FIXTURES_ROOT`, `HC_MEMO_TIMEOUT_MS`. + +Breadth comes from `SIL.Machine.Morphology.HermitCrab.Conformance.Fixture.DiscoverAll(root)` over +`conformance/{languages,edge-cases}` — 33 grammars, each `grammar.xml` + `words.yaml` with +hand-derived expectations. Depth comes from the three real corpora. **Amharic: never exceed 28 +words** — word 29 (`ሌባዎቹ`) is pathological and burns hours. + +Run the memoized path: `new Morpher(traceManager, language, maxDegreeOfParallelism: 1)`. Tracing +disables the memo, so a traced run measures nothing. + +### A/B methodology, all of it earned by being wrong first + +Parity is the gate and speed is only reported. Acceptance is **analysis-set equality on canonical +morpheme-signature sets** (`MorpherTests.WordAnalysisSignature`, sorted) — never object or byte +equality, because a replayed `Word` is legitimately not field-for-field identical. + +Timing: **discard a warm-up rep per arm, then take min of N interleaved samples, and print the +off-arm spread beside every speedup.** Refuse any speedup smaller than its own spread. Three +failures forced each part of this — a first A/B read 1.53x that was entirely JIT; another read +2.385x with a 210.9% spread on one arm that was 1.015x serial; and adding one arm's warm-up while +discarding the other's manufactured a uniform ~2.4x across every fixture including ones with zero +shareable steps. + +**Never dispatch timing-sensitive probes concurrently.** Three at once put four testhosts on the +machine and invalidated three rows' bucket percentages. Check `tasklist | grep -c testhost` before +any timing run and report the count alongside the numbers. Counts survive contention; shares and +speedups do not. + +**Never aggregate the test logger's console output.** It prints every line twice, the second copy +space-prefixed, so a `^\s*` grep doubles every total — this published a 2x error. + +Sub-50 ms fixtures cannot carry a timing claim: one 10 ms fixture's share moved 1.3% → 14.5% +between two runs of identical code. Flag them (`reliable = wallMs >= 50`) rather than filtering them +out silently. + +### One concurrency hazard worth knowing + +A per-word timeout wrapper built on `Task` has no cooperative cancellation, so an abandoned word's +task keeps mutating engine state after the harness moves on. Harmless with a per-word +`AnalysisScope`; under a **shared** scope it collided with the next word on the same +`Memo`/`InProgress` collections and produced `IndexOutOfRangeException` inside +`MemoizedCombinationRuleCascade`. A shared-arm timeout must abort the run, not continue. Also note +`Task.Result` throws `AggregateException`, not the bare exception type. + +--- + +## 4. Two durable engine facts, restated here so they survive independently + +**`Word.ReplayOnto` does not splice `_mrulesUnapplied`.** It splices `_mruleApps` and `_nonHeadApps` +only. This is safe *solely* because `AnalysisStateKey` includes the count multiset, which guarantees +a memo hit has identical arrival and stored-arrival counts. **Narrowing that key breaks correctness +with no test failure.** Fix, if the key ever changes: store arrival counts on `MemoEntry` and compute +`stored - storedArrival + query` — a no-op under today's key, which is also how to test it. + +**`AnalysisScope.MaxMemoEntries` is a per-scope cap with no eviction.** Sharing one scope across a +corpus therefore makes words compete for a fixed budget: two heavy Sena words exhausted it at index +74 and 76, and every later word lost the within-word memoization a fresh scope would have given it. +Cross-word hits went *negative* and the corpus ran 28–32% slower. A shared bounded cache without +eviction is not a smaller version of the optimization.