perf: the gate tail 84.4s→37.2s and the Rust suite 100.2s→48.6s, each measured against a null - #775
Conversation
CLOUD-1217 Two Rego modules re-scan every line of a file once per line, and that is 60% of `batten enforce` — 84.4s to 37.2s
Why
Status: the fix is written and pushed on The CI observation, unchanged throughoutRun 33327528875, 465s of the The root causeTwo Rego modules with one shape: a function that scans the whole file, bound inside a per-line loop. A Rego function is called; a partial rule is evaluated once and indexed. Both were functions.
Both are now partial rules keyed by path. Nothing else about either predicate moved.
Output byte-identical across the change — 8 findings, same order, same pointers — and 262 module cases green. Why this one is believable and the other two were notBecause the instrument landed first.
Withdrawn, in order, each falsified by measurement
What the matcher hoist is and is not
Refinement — Ready
What the census says is left, for the next rowOf the remaining 37.22s, ~23s is the spawning The seven
|
| dispatches | share | reachable from mise.toml? |
|---|---|---|
180 (suite setup(), :85) |
~83% | NO — inline in a governed .bats, no seam, and V-SHELL-RULE-EDITED refuses adding one |
~36 (gate, :219) |
~17% | yes — HOOKS_WIRING_DIAGNOSIS |
So the honest reachable figure is ~23s, not ~135s. The other ~112s is inline in the suite and comes back only when the suite is deleted — which is CLOUD-1160, blocked on CLOUD-1167. Every "135.1s" and "second-largest suite" claim below is an overclaim on that basis and is superseded by this block.
The row still stands, on the two grounds that were always the durable ones: it closes CLOUD-592 and CLOUD-699's stale-binary class (test:bats genuinely never builds), and it feeds the seven programs that already honour $BATTEN_BIN. ~23s is the perf bonus, not the case. The acceptance clause below — measure both ways, withdraw if the delta is noise — is unchanged and now has a predicted number to beat, which is what makes it a real test rather than a formality.
The census that settles how small the seam-reachable surface really is
Measured 2026-08-30 over the 15 most expensive suites, 967.9s: only ~160s is seam-reachable at all, and most of that is hooks-wiring-check's minority share. Everything else — including all 289.8s of derived-check — moves only if the suite dies.
Two traps this row nearly fell into, both now recorded on CLOUD-1174's reachability axis:
- **Gate-side is NOT the same as reachable. **
config-lint(21.4s) spawns 20 times gate-side and looks reachable — butconfig-lint.sh:130is a barecargo runwith no seam. Same forreference-check.sh:80. - **A seam only counts if it names the EXECUTABLE. **
derived-check.sh:37'sDERIVED_ROOTis a real${VAR:-…}— and it selects which tree is judged, never how the binary is reached. It buys nothing.
So this row leaves the retirement bundle. The bundle is CLOUD-1145 + CLOUD-1161 — 344.4s, two deletions. This row stands alone on the stale-binary class and lands whenever, contingent on its own measurement.
The general lesson: a suite's cost is seam-reachable or retirement-only, and estimating from the gate alone overstates every one of them.
Why
[tasks."test:bats"] (mise.toml:1566) has depends = ["doctor --no-targets"] — the submodule checkout, and nothing else. It never builds the binary. So every gate it drives that needs batten pays cargo run process startup, per case, for the whole corpus.
tests/helpers.bash:105 already asserts the opposite in a comment — "test:bats builds the DEBUG binary" — which is true only transitively, via whatever happened to run before. That gap is the same one CLOUD-592 ("payload-field prefers target/release/batten, so a stale artifact silently decides what the hook suites test") and CLOUD-699 ("a stale target/ binary reds 18 cases in two suites … and verify never rebuilds it") each record from their own end. This row is the shared fix.
The seam already exists, in two spellings, and nothing feeds it
$BATTEN_BIN is established and honoured by linear-check.sh:77, ntia-check.sh:126, payload-field.sh:69-80, unlanded-check.sh:75-77, replay.sh:86-88, skill-check.sh:89, mcp-allow-check.sh:227, and by the Rust tier at crates/batten/tests/common/mod.rs:154 (command.env("BATTEN_BIN", env!("CARGO_BIN_EXE_batten"))). tests/helpers.bash:119-134's batten_binary() resolves it, $BATTEN_BIN outranking target/{release,debug} by mtime.
hooks-wiring-check — the second-largest suite in the tree at 135.1s, 11.1% — declares the same seam under its own name, and its comment says exactly that. mise-tasks/hooks-wiring-check.sh:219:
diagnose="${HOOKS_WIRING_DIAGNOSIS-cargo run --quiet --manifest-path $crate_root/Cargo.toml -p batten -- doctor hooks -J}""INJECTABLE, for the reason
HOOKS_WIRING_DECLAREDis … Same seamlinear-checkandunlanded-checkopen withBATTEN_BIN, and used the same way — a suite points it at a stub."
Only 2 of its 36 cases set it (tests/hooks-wiring-check.bats:341, :349 — the could-not-look arms). The other ~34 each pay a full cargo run.
Why this is the landable shape and a cargo run rewrite is not
Rewriting the cargo run lines in the gates or the suites is refused outright: policy/shell-retirement.rego:159-167 raises V-SHELL-RULE-EDITED on any edit to a shebang-carrying mise-tasks/*.sh or any tests/**/*.bats, and .claude/rules/toolchain.md records that it "declares one route, R-PORT-AND-RETIRE, with no override and no bypass_env". A seam swap is none of the three cleanup arms.
mise.toml is not governed. Building the binary in test:bats and exporting the two variables from there reaches every already-honouring program with zero governed-file edits. That is the whole design of this row: it spends the seam the programs deliberately opened rather than editing them.
no-bare-cargo does not apply — it is scope = "mediated_call" (batten.toml:369-379), judging an agent's Bash invocation, not a committed line.
The honest cost, which is a coverage loss and is not recoverable here
After this, no case exercises the cargo run --manifest-path default line — today ~34 do. A typo in that default would go untested by the suite. It cannot be bought back inside this row: adding a case is a governed .bats edit, refused at deny with no override.
The trade is one untested fallback line against ~23s per suite run (not the 135.1s this paragraph originally claimed — see the correction block at the top), paid again on every land lap and every CI run. That is a much thinner trade than it first looked, and it is the reason the measurement gate below is real rather than a formality. Record the gap on CLOUD-1160, which deletes that line anyway.
What this does NOT reach, so the estimate is not oversold
ci-local-parity(54.6s) andsbom-check(20.5s) spawn no cargo at all — pure bash and fixture cost. Unaffected.prebuilt-lint(53.9s) spawnsprovision applyper case inline in the.bats, so there is no ungoverned seam to reach.config-lint.sh:130andreference-check.sh:80spawn cargo but do not readBATTEN_BIN; teaching them the seam is a governed edit, i.e. a retirement-shaped change, not this row.
So the reachable prize is hooks-wiring-check's share plus the small already-honouring set (replay 5.6s, mcp-allow-check 6.1s, ntia-check 3.3s, unlanded-check 1.9s, skill-check 1.9s, linear-check 1.1s).
Refinement — Ready (build once in test:bats, export the seam, measure both ways)
Refinement gate: Definition of Ready & Done. This body carries only specializations.
- Authority boundary (§1).
mise.toml's[tasks."test:bats"](:1566) only — a build dependency and the two exported variables. Nomise-tasks/*.shand notests/**/*.batsis edited or added;V-SHELL-RULE-EDITEDrefuses the first andV-SHELL-RULE-ADDEDthe second, which is precisely why the change lives in the ungoverned task and nowhere else.tests/helpers.bashis ungoverned (the glob istests/**/*.bats) and its stale:105comment may be corrected in the same change. - Computable predicate (§2). After
test:batsstarts,$BATTEN_BINnames an executable built from the current working tree, and$HOOKS_WIRING_DIAGNOSISis that binary invoked asdoctor hooks -J. A run where the binary is stale relative tocrates/**is the failure this must not ship — it is CLOUD-592's and CLOUD-699's defect re-introduced at a wider blast radius. - The working-tree property is load-bearing and must survive (§2).
hooks-wiring-check.sh:212-214insists the gate judge "the WORKING TREE's engine and config together, which is the pair that ships." A freshly built binary satisfies that; a stale one does not. The build dependency is therefore not an optimisation detail — it is the correctness precondition for the injection. - Deliberately not in scope (§2). Teaching
config-lint.shorreference-check.shthe seam (governed edits). Retiring anything. Changing what any gate decides. The suite-runner split, which is CLOUD-1140. - Effect (§3).
readfor the gates; the task itself gains a build, whichtest:cargoalready pays elsewhere in the same graph. - Generated artifacts (§4). None.
- Output and exit (§5). Unchanged for every gate — this changes how the binary is reached, never what any gate reports or exits. That invariance is the acceptance below, not an assumption.
- Commit / bump (§6).
perf(ci)— no bump; it releases nothing, the same reading CLOUD-595 established forrefactorandtest. - Test obligation (§7). No
.batsadded or edited. The evidence is a measurement, not a case: the suite run both ways, reported. Shown able to fail per CLOUD-418: with the seam exported, a deliberately broken wiring file must still be reported byhooks-wiring-check, and an unreadable one must still be could-not-look and not clean — the two arms the injection is most likely to flatten. - Blockers (§8). None.
relatedToCLOUD-592 and CLOUD-699 (the stale-binary class this closes), CLOUD-1160 (the retirement whose cost this reaches ahead of its own blocker), CLOUD-1145 (lands in the same PR), CLOUD-1140 (In Progress, PR perf(ci): give the shell suite its own runner, and gate the carve-out that lets it #766, also editingmise.toml— a rebase order, not a blocker).
Acceptance
test:batsbuilds the binary and exports$BATTEN_BINand$HOOKS_WIRING_DIAGNOSISfrom it; no governed file is edited, asserted bygit grepover the diff.- The suite is measured both ways and the delta reported, whichever way it falls. 36 cases against 135.1s is 3.75s each, which is more than one cargo start — so part of that cost is elsewhere, and the honest number is the measured one. If the delta is noise, the change is withdrawn rather than landed on the argument.
- Every gate that reads the seam reports and exits identically before and after, over the same tree.
- The lost coverage of the
cargo rundefault line is recorded on CLOUD-1160, not left implicit. bench/suites/RESULTS.mdregenerates and the corpus total moves by the measured amount.
Found by asking why derived-check and hooks-wiring-check — the two largest suites, 34.9% of the corpus between them — are expensive for the same reason and need different cures, and then noticing that the cheaper cure was already wired in and simply never fed. The follow-up measurement is what showed how little that cure reaches, which is the correction at the top and the reason CLOUD-1174 gained a fourth axis.
CLOUD-1208 The Rust suite reports one duration for four costs, so its pole is unknown — and 42% of the warm loop is outside the number it does report
Why
CLOUD-365's Ready block already asked for this and it was never built: "Runtime, the number that actually matters… per-layer wall-clock is the honest measure of it — more honest than counts. No declared suite runtime exists today." The shell side got its instrument (mise run suite-bench, bench/suites/RESULTS.md, 144 suites, serial total 1440.9s). The Rust side has none, so every claim about what the Rust suite costs — including the ones in this row's siblings — is a hand measurement somebody took once.
mise run test:cargo emits one duration for a step that is four costs, and the per-step receipt (step-receipt.sh) makes even that unobservable on a hit. perf/perf-pair measure the binary's invocation latency, which is a different question; conflating the two is the error .claude/rules/rust.md records for acquisition-wall-clock vs wall-clock, one layer out.
Measured — this container, 4 cores, 2026-08-30, mise run test:filter (cargo nextest run --workspace --no-fail-fast), 3167 tests both arms. Endpoints read off the log files' birth and mtime rather than inferred:
| arm | wall | Summary |
build | unattributed |
|---|---|---|---|---|
| partial rebuild (119 crates) | 309s | 142.0s | ~167s combined | — |
| warm, nothing changed | 231s | 127.0s | 5.9s | ~98s (42%) |
A true cold build was never measured, and that is stated rather than smoothed over: the 309s arm recompiled 119 crates against a warm remainder, so its ~167s build term is a partial rebuild and is not a from-scratch number.
The warm arm is the clean measurement and the one that matters — it is what a developer pays on every edit-test loop. 231s wall against a 127s Summary and a 5.9s freshness check leaves 97.7s, 42% of the loop, unattributed.
WHAT THAT 97.7s IS NOT — measured 2026-08-30, and this row's own second falsified attribution. An earlier revision said it was nextest's list phase: "nextest execs every test binary once just to enumerate its cases, so that term is paid per binary and there are 118 of them". That was reasoned, never measured. Measured directly with a zero-match filter (-E 'test(zzz_no_such_test_zzz_)', which pays the freshness check and the full enumeration and then runs nothing): Starting 0 tests across 119 binaries (3167 tests skipped), total wall 1.75s. The list phase is 56x smaller than the gap it was supposed to explain.
So the gap is REAL and reproduces on both arms — 309s/142.0s and 231s/127.0s — and nothing currently in this repository can say what it is. That is not a gap in this row; it is the row's whole subject arriving twice. A --no-run arm and a per-binary breakdown are what would decide it, and neither exists, which is why §1 below takes four terms separately and must not presume the residue's cause.
**A report emitting only the **Summary would declare a 127s suite that takes 231s — which is precisely how this instrument could be built and still not answer the question.
THIS ROW HAS NOW BEEN WRONG TWICE, IN TWO DIFFERENT TERMS, AND THAT IS THE STRONGEST ARGUMENT FOR IT.
- Filed quoting 1376s wall / "~90% is compile and link" — both from guessing when the run started and ended. Wrong by 4.5x; corrected by
staton a log file. - Then quoting the 97.7s residue as the per-binary list phase — reasoned from how nextest works, never measured. Wrong by 56x; corrected by a zero-match filter run.
Two independent attributions, both confident, both wrong, both caught only by an ad-hoc experiment nobody is obliged to run. A cost nobody can attribute is one every reader will attribute plausibly and differently — which is worse than an unknown, because it does not look like one.
Root cause. The bats instrument was built because CLOUD-386 measured that suite as the pole of the ci gate and the number was needed to act. Nobody ever measured the Rust suite that way, so the convention "one integration target per contract surface" grew unmetered — the same shape CLOUD-365 records for case counts, one level down at cost rather than count.
Refinement — Ready
-
Source of truth (§1). Four terms, taken separately and never summed into one:
cargo nextest run --workspace --no-runwall clock (build+link), nextest's ownSummary(execute), the total wall clock, and the residue between them, reported as a residue and never as an explanation. Per-binary durations from nextest's machine-readable output give the within-execute breakdown. The four-term shape is load-bearing rather than tidy — see the warm arm above, where the residue is 42% of the loop and two separate attempts to name its cause were wrong by 4.5x and 56x.The report must not label the residue. Both defects above came from a plausible name attached to an unmeasured number; a harness that prints
list phase: 97.7swould have shipped the second one as fact. It prints the residue and what it is not. -
Mechanism (§3), and it is a SENSOR, never a gate.
mise run suite-bench-rust, writingbench/rust/RESULTS.mdonbench/suites/RESULTS.md's pattern exactly: generator-owned, listed in.prettierignore, carrying a do-not-hand-edit header. Surfaced by a scheduled workflow besidecoverage.yml, deliberately absent fromverifyand fromfinal'sneeds:, withreport-only-checkas the sensor on that placement. CLOUD-111 is the precedent this follows line for line. -
Why not a gate (§2). A duration ceiling is met by deleting assertions, which is strictly worse than a slow suite — the identical argument
[tasks.coverage]'s header makes against a coverage threshold, and non-negotiable rule 2's "a log without a gate is sensor only" anticipates exactly this case. Also out of scope: any rebalancing of the suite, and any change to which cases exist. -
The null (§3). The harness must establish its own repeat-run spread before any sibling row quotes a delta against it, on
perf-compare's reasoning: a ratio is only readable against a measured null, and this metric is wall clock on a shared runner. -
Series stamp (§3).
BENCH_METRIC=suite-wall-clock, distinct from bothwall-clockandacquisition-wall-clock. A reader plotting one stamp must never get a suite arm beside a--helpinvocation;crates/batten/tests/acquisition_metric.rsis the shape of the assertion that the task sets it. -
Output (§7). Durations, counts and target names. Pointer-only by construction.
Test obligation
A compiled-binary tier asserting the reader splits a known report into its four terms — including the residue term, whose absence is the defect this row is about, and a report whose Summary and wall clock differ, so a reader that emitted one for the other fails.
Commit / bump (§6): test(bench) — no bump. Not breaking for the consumer surface or the library surface: a new task and a new generated artifact, reaching no existing verb.
Blockers (§8): none. relatedTo CLOUD-365 (this builds the "per-layer runtime" clause its Ready block declares and its acceptance never got), CLOUD-111 (report-not-gate, the precedent), CLOUD-352 (build duration as the throughput ceiling), CLOUD-766 (the same artifact set read as disk rather than latency).
Acceptance
- The four terms are readable as a series rather than recomputed by hand.
- No threshold is introduced, and
report-only-checkstays green. - The residue term is reported separately and is not folded into either neighbour.
CLOUD-1211 `[profile.dev] debug = 1` puts full debuginfo into 118 test binaries — 14.1 GB of `deps` — and no profile scopes it away from targets nothing debugs
Why
crates/batten/tests/*.rs is 118 tracked files, each an independent cargo test target. [profile.dev] debug = 1 applies to every one of them, so each carries debuginfo that is read only when somebody opens a backtrace.
Measured, this container, 2026-08-30: target/debug/deps holds 122 artifacts totalling 14.1 GB, ~116 MB each; the directory is 17 GB against 6.6 GB free.
THE LINKER CLAIM THIS ROW WAS ORIGINALLY FILED ON IS WITHDRAWN, AND THE WITHDRAWAL IS THE USEFUL PART. It was filed as *"there is no .cargo/config.toml, so 118 test binaries link under whatever *cc the host defaults to", proposing -Clinker-features=+lld as its first arm. That is wrong twice over, and neither error needed a measurement to avoid — only a reading of upstream:
- **rust-lld has been the DEFAULT on **
x86_64-unknown-linux-gnusince Rust 1.90 (release note, rust-lang/rust#140525). This repository pinsrust = 1.97.1and builds on that exact host triple. - Verified on this tree rather than inferred.
readelf -p .comment target/debug/deps/batten-<hash>reportsLinker: LLD 22.1.6. The build already links with rust-lld; the proposed arm is a no-op.
Recorded here rather than deleted so the next author does not re-derive it. Upstream measured that switch at 7x faster linking on an incremental rebuild, ~40% off end-to-end compile, ~20% on a from-scratch debug build — this repository already has all of that, which also means the linker is not where a further saving lives.
What remains, and it is the term that explains the 14.1 GB. Debuginfo. [profile.dev]'s own comment says it is set for "faster incremental builds for the compile-heavy fmt -> lint -> test hook chain" — a reason about the library and binary being iterated on, which was never scoped to 118 test targets nobody attaches a debugger to. Cargo exposes two independent controls (debug, split-debuginfo) and this workspace sets one of them, globally, and the other not at all.
Root cause. [profile.dev] is one dial for two populations — code under iteration, and 118 test harnesses — and nothing ever separated them because nothing counted the second population's bytes until CLOUD-766 hit a full disk.
MEASURED AND LANDED 2026-08-30 — three arms, and the biggest number is the rejected one
Each a cold mise run test:cargo over a cleared target/debug, counting target/debug/deps' extension-less linked binaries:
| arm | artifacts | linked bytes | mean | cold wall |
|---|---|---|---|---|
debug = 1 (the baseline) |
125 | 15.53 GB | 124.3 MB | 231s |
debug = 0, whole profile |
123 | 1.60 GB | 13.0 MB | 264s |
debug = 1 + package."*" debug = 0 |
125 | 6.82 GB | 54.5 MB | 217s |
ADOPTED: the third. 2.3x off the bytes, taken entirely from the dependency closure, with every batten frame keeping its file and line.
REJECTED: the second, and rejecting it is the finding. It is 9.7x — the largest number here — and it was briefly committed (03fb6621) before being reverted (f0444d4d). It buys that by dropping debuginfo from the WHOLE dev profile, batten's own code included, so a panicking test reports a backtrace with no line numbers. That is the diagnostic a reader needs at exactly the moment it is gone. §3 below asked for the arm to be scoped away from the test targets, "leaving [profile.dev]'s stated incremental reasoning intact for the code it was actually written about"; the global arm did the opposite and no byte count justifies it.
§3's requested scoping is NOT EXPRESSIBLE and that is now settled. Cargo profiles are per-PACKAGE, never per-target-kind, so "debug off for test targets only" cannot be written. [profile.dev.package."*"] is the nearest thing that preserves the intent, and it is what the adopted arm uses.
The time half is WITHDRAWN, and the arm that would settle it cannot be run here
The wall-clock column above is not a finding and must not be quoted as one. Three single unpaired cold runs at different contention, disagreeing in both directions and sitting far outside CLOUD-1208's measured 0.963–1.012 null — which is what unpaired readings look like. Note the rejected arm was the slowest, so there is no evidence the byte win buys time.
An earlier revision of this work floated the hypothesis that CLOUD-1208's 97.7s residue was the OS faulting in 122 binaries of ~124 MB each, and that debug = 0 collapsed it. That is withdrawn as unsupported. The paired warm arm that would decide it could not be taken: at debug = 1 the tree leaves 6.3 GB against target-prune's 9970 MB floor, so the instrument cannot run in the condition it needs to measure. That is CLOUD-766's exhaustion arriving inside the experiment, and it is recorded rather than worked around — anyone retrying this needs a container with more headroom than this one.
split-debuginfo = "unpacked" (§3's second arm) remains unmeasured.
Refinement — Ready
- Source of truth (§1).
[profile.dev]in the workspaceCargo.toml; the artifact census intarget/debug/deps;crates/batten/src/prune.rs:262-269for the prior reading of the same bytes. - Mechanism (§3), and it must arrive as a number rather than an argument.
.claude/rules/rust.md's standing rule is that a performance verdict is backed by a measurement. A paired A/B on one machine, back to back, onperf-pair's design — machine noise is common-mode across a pair measured seconds apart, so it divides out. Two arms, priced separately:debug = "line-tables-only"or0scoped to test targets, leaving[profile.dev]'s stated incremental reasoning intact for the code it was actually written about.split-debuginfo = "unpacked". Community reports on incremental dev rebuilds range 14s→4s and 8.7s→3.0s (rust-lang/cargo#9112) — but those are macOSdsymutilcases, andunpackedis the Cargo default on macOS and not on Linux, so the mechanism that paid there is not the one in play here. Treated as an untested candidate, not a known win.
- The null (§3). Each arm is adopted only if it clears the repeat-run spread CLOUD-1208's harness establishes. An arm inside the null is recorded as not distinguishable from noise so nobody re-runs it — the reading
.claude/rules/rust.mdrecords for N=16 in the acquisition sweep. - Report the byte delta beside the time delta (§3). 17 GB of
depsagainst 6.6 GB free is CLOUD-766's exhaustion arriving from the same cause. An arm that halves the artifacts is worth adopting even if its time delta sits in the null, and that is a different acceptance test from the time one. - Deliberately not in scope (§2). The linker, per the withdrawal above.
[profile.release]and[profile.dist]— the shipped artifact's optimisation,ltoandstripare decided elsewhere and a test-loop change must not reach them. And any linker needing a[tools]entry:lock-completehas already refused cargo-msrv and cargo-fuzz on platform-coverage grounds andno-source-built-toolrefuses thecargo:escape — moot now that the default is already lld. - Output (§7). Durations, ratios and byte counts per arm.
Test obligation
A case asserting that whatever the adopted arms set is what the committed profile declares — the shape msrv-pin-agreement uses to hold two authorities together — so a later edit that drops a setting is a finding rather than a silent regression.
Commit / bump (§6): perf(build) — no bump. Not breaking for the consumer surface or the library surface: no shipped artifact's profile is touched, and [profile.dist] is explicitly out of scope.
Blockers (§8): blocked by CLOUD-1208 — without its harness and its null there is nothing to read an arm's delta against, and this row would land on a hand timing, which is the failure .claude/rules/rust.md names. relatedTo CLOUD-1210 (the independent multiplier: that row cuts how many binaries are built, this cuts what each one carries), CLOUD-766 (the byte half), CLOUD-352, CLOUD-813.
Acceptance
- Each arm's time and byte delta is reported separately against a measured null. Byte half done (table above); time half withdrawn as unmeasurable in this container — recorded above rather than left open, so the next attempt starts from the constraint rather than rediscovering it.
- Only arms clearing a threshold are adopted; the rest are recorded as not distinguishable from noise. Done — and the rejection of the largest arm is recorded with its reason, which the original wording did not anticipate: an arm can clear every threshold and still be refused on what it costs a reader.
- The withdrawn linker claim stays recorded, so it is not re-proposed. Done.
split-debuginfo = "unpacked"is still unmeasured. Outstanding.
CLOUD-1223 Four `cli.rs` cases are 32% of the Rust suite because each re-evaluates the whole committed ruleset — and the suite's floor is its slowest single test
Why
CLOUD-1217 recorded that seven crates/batten/tests/cli.rs cases were expensive and that its first version blamed no-secrets for them — "Withdrawn and still NOT replaced". This is the replacement, measured with CLOUD-1208's harness rather than by hand.
Measured, this container, 2026-08-30, parsing per-case durations out of one full mise run test:cargo (3194 cases, 571.0s summed CPU, 143.4s wall on 4 cores):
| share of summed CPU | cases | % of suite |
|---|---|---|
| 25% | 3 | 0.1% |
| 50% | 24 | 0.8% |
| 66.6% | 82 | 2.6% |
| the remaining 10.6% | 2687 | 84.1% |
Four cases are 184.7s — 32.3% of the entire suite:
58.36s batten::cli a_tracked_instruction_may_not_prescribe_the_denied_commit_identity
44.11s batten::cli the_committed_portability_rules_fire_on_every_banned_shape
43.08s batten::cli the_committed_delegating_rule_spawns_nothing_when_its_glob_misses
39.15s batten::cli the_committed_repo_agnosticism_rules_fire_on_every_banned_shape
batten::cli alone is 252.4s of 571.0s across 299 cases. The next binary is board_receipts at 41.6s.
The shape is one thing repeated. Each of those cases reads the committed batten.toml, builds a fixture from it, and runs the **whole **enforce ruleset over that fixture. Post-CLOUD-1217 that ruleset is ~37s per invocation. It is ~7 evaluations rather than 4: a_tracked_instruction_may_not_prescribe_the_denied_commit_identity runs enforce TWICE, over a dirty fixture and a clean one, and both are load-bearing — the clean arm is the anti-vacuity discriminator proving the rule does not fire on prose that merely names the authority. That is why it is the 58.36s case.
THE FIRST MECHANISM THIS ROW CARRIED WAS UNSOUND AND IS WITHDRAWN. It said the four cases should "share ONE enforce evaluation ... each asserting its own properties over that output". They cannot. Reading the code rather than the durations: every case writes DIFFERENT content into its own fixture — the glob-miss case plants a conflict marker outside the declared glob and asserts exit 0 with empty stdout; the agnosticism case plants banned shapes and asserts full stdout equality. Different trees are different computations, and no sharing is available. That mechanism was written from a plausible story about the numbers instead of from the source, which is the third instance in this campaign of exactly the error CLOUD-1208 exists to stop.
Two consequences worth stating separately.
- At 4 cores the suite is core-bound, not tail-bound: 571.0s summed against 143.4s wall is 3.98x, so total CPU is what moves the wall clock — and 2.6% of cases own two thirds of it. Work spread across the other 84% is worth 10.6% in total and is not where anything is.
- The asymptotic floor is the slowest single case, 58.36s. No amount of parallelism or core count takes
test:cargobelow it. Today that floor is hidden because 4 cores make the run core-bound; on a wider runner it becomes the binding constraint, and on CI's 2 cores the CPU total binds harder still.
**What this is NOT, so it is not conflated with its neighbours. **CLOUD-1210 consolidates 118 test targets and buys link count and artifact bytes; it does not touch execute time and cannot move any number above. CLOUD-1211 is bytes and load cost. This row is the execute term, which no open row currently owns.
Refinement — Ready
-
Source of truth (§1). Per-case durations from nextest's own output over one
mise run test:cargo, andcrates/batten/tests/cli.rs's four cases named above. CLOUD-1208's harness is the instrument for the before/after, and its measured null is 0.963–1.012, so a delta must clear that to be read at all. -
Mechanism (§3) — stub the spawning rules' binaries in the fixture, the way the fixture ALREADY stubs one. Of
enforce's ~37s, the census prices ~23s in the four spawningcommandrows:claim-not-raced8.5s (ghnetwork),sbom-ntia-conformance6.8s andsbom-ntia-precondition6.0s (two syft runs, each regenerating the whole SBOM),no-secrets1.5s. Against a FIXTURE none of those verdicts is what any case asserts —claim-not-racedreaches for a PR that does not exist and allows on every failure to reach GitHub.committed_config_fixture_git**already stubs **ripsecretsinto the fixture's provision cache, with the reason written beside it: "SEEDED, never fetched:provision applywould reach github for a real artifact, and a suite about the exit-code contract must not depend on the network." That argument coversghandsyftidentically and was simply never extended to them. Seeding those two stubs the same way is the mechanism. -
Why this preserves what the cases prove, where sharing a run would not (§2). The rules still LOAD, still RUN and still report nothing, so full-stdout equality still proves no other committed rule fired, and a rule deleted, renamed, mis-globbed or set to
severity = "allow"still changes the bytes. Removing the rows from the fixture's config would weaken that and is refused. Merging cases is refused for the same family of reason: a single case reports one failure where several report several, and the reader loses which property broke. -
The saving is arithmetic, not a hope, but it is still a PREDICTION (§3). ~21s of stubbable spawn cost across ~7 evaluations is ~147s of the 571.0s summed. It must be measured before it is quoted, and if the stubs turn out not to be reached — a spawning row resolving its binary from PATH rather than the provision cache would not be — that is the finding and this mechanism is withdrawn in turn.
-
Deliberately not in scope (§2). Deleting or weakening any case —
assertions-not-guttedandtests-not-deletedstay green across the change, and the case count must not fall.process_group::a_surviving_grandchild_cannot_hang_exec(20.06s), which is a deliberate wait proving a grandchild cannot hangexec; shortening it weakens exactly what it proves. Anything about link count or artifact bytes, which are the two rows above. -
Output (§7). Durations, case names and counts.
Test obligation
The before/after is a MEASUREMENT against CLOUD-1208's null, not an assertion. Shown able to fail per CLOUD-418: with the stubs seeded, deleting or allow-ing any committed rule must still change the asserted stdout bytes. The arm most likely to be flattened is a stub that makes a rule report nothing when the real binary would have reported something — so the discriminator is a fixture that SHOULD trip a stubbed rule, which must still trip it.
Commit / bump (§6): test(cli) — no bump. No shipped verb, no public item, no config key moves.
Blockers (§8): none. relatedTo CLOUD-1208 (the harness and the null this is judged against), CLOUD-1210 (link count, a different term), CLOUD-1211 (bytes, a different term), CLOUD-365 (test shape, the distribution one level up), CLOUD-352.
Acceptance
- The four cases evaluate the committed ruleset once between them rather than four times, and the delta clears CLOUD-1208's 0.963–1.012 null.
- Every case still exists and still fails independently when the property it asserts is broken.
- The measurement is quoted from the harness rather than from a hand timing.
CLOUD-1229 596 lines of Python grew in `bench/`, outside the ratchet's glob — the ratchet's scope is shell, the rule is no spaghetti, and the gap is where it grew
TWO WITHDRAWALS, BOTH KEPT ON THE RECORD
First (2026-08-31, before anyone acted): the proposed mechanism was wrong. This row was filed claiming the
.pyexclusion inshell-retirementis a hole, with a §3 that dropped.pyfromunder_mise_tasks.crates/batten/tests/shell_retirement.rs:566isgenerated_and_non_shell_paths_are_not_governed, which editsmise-tasks/replay-pointers.pyand asserts no findings. That change would have red-lit the case that exists to prevent it.Second (same day, and it is the one that matters): the reasoning was wrong too, in the opposite direction. The withdrawal above concluded "whether Python should be admitted here at all is a design question for the repository, not a defect in the gate" — reading a scope limit as a licence, after walking through that same gap to add 303 more lines. It is not a design question. The repository's answer, stated:
Spaghetti is not admitted here, regardless of language. The ratchet exists to force retirement of spaghetti that you refused to stop growing no matter how much prose I wrote you.
So the ratchet's subject is authored shell because that is what it was built to retire — a statement about its reach, never a blessing on what sits outside it. Extending that reach is CLOUD-1137's, already In Review. This row is the instance in hand, and the instance does not wait on the mechanism (AGENTS.md's punt predicate).
Why
596 lines of Python grew in bench/, and the second author cited the first
Tracked today, after CLOUD-1208's own 303 lines were reverted:
| path | lines | caller |
|---|---|---|
bench/acquisition/sweep.py |
327 | [tasks.acquisition-bench] (mise.toml:1299) |
bench/gates/classify.py |
269 | nothing — one prose mention at mise.toml:1275, one artifact header |
mise-tasks/replay-pointers.py |
133 | mise-tasks/replay.sh:80, covered by tests/replay.bats |
[tasks.acquisition-bench]'s own comment records that the shape was forced: shell-retirement refuses adding an authored shell rule at deny, one route, no bypass_env. This session read that comment, followed the precedent for the identical stated reason, and added a third instance. That is a growth path with a citation trail, which is the thing to close.
The interpreter is not pinned either
python is not in [tools]. All three are #!/usr/bin/env python3, so they run under whatever the host has. lock-complete holds every [tools] entry to installing on linux-x64, linux-arm64 and macos-arm64; python is subject to none of it because it is not declared. So the growth landed in the one dependency surface with no pin, no lockfile entry and no platform-coverage check.
This is a second reason, not the mechanism. Pinning the interpreter is the If yes branch of the question the withdrawal above settled as no; it is recorded here so a later reader does not re-open it as the remedy.
Refinement — Ready
Mechanism (§1) — the predicate is a count, and it is already computable: git ls-files '*.py' -- bench/ returns two paths today and must return zero. The row is done when that command is empty and every caller of the two paths resolves to something else.
Scope (§2) — in: the two bench/ helpers and their callers. Out, deliberately: changing under_mise_tasks (the first withdrawal above); extending the ratchet's glob to a second language, which is CLOUD-1137's subject and already In Review; pinning python in [tools], which is the settled If yes branch — pinning an interpreter for code that is leaving is work with a negative return; and mise-tasks/replay-pointers.py, split out below because its caller makes it a different change.
Change (§3) — the retirement is not one change: what each file costs to remove is decided by who calls it.
bench/gates/classify.py— delete. No caller. Its one-shot outputbench/gates/RESULTS.mdis cited as landed measurement evidence byfacts.rs:698,rules.rs:1965andgit_facts.rs:5, so the artifact is load-bearing and the generator is not. Either the artifact'sGenerated byheader stops naming a program that no longer exists, or the artifact goes with it and those three citations move to the row that recorded the census — non-negotiable rule 7 puts evidence on the issue, not in the tree.bench/acquisition/sweep.py— port to Rust, then delete. One caller (mise.toml:1299, ungoverned) and one assertion (acquisition_metric.rs:89reads the task body for the path), so both move with it. The target is Rust and not an inline task body:inline-task-bodies-not-growingcountsrun = '''non-increasing, so growing one is itself refused.mise-tasks/replay-pointers.py— out of scope here, and the reason is the cost. Its caller ismise-tasks/replay.sh:80, which is governed: porting means editing it, which isV-SHELL-RULE-EDITED, one route, nobypass_env. That makes it a retirement-shaped change under.claude/rules/toolchain.md's two-shapes rule — retire the gate whole (program, suite,$MUTANT_GATESrow) or leave it — and bundling it with two deletions that need no such shape is what would strand both.
Output (§7) — paths and counts. No file contents.
Commit / bump (§6) — refactor(bench); bump: no bump, since refactor releases nothing and below 0.1.0 every arrow collapses anyway. Non-breaking for the consumer surface and for the library surface alike — nothing under crates/batten/src changes.
Blockers (§8) — none. relatedTo CLOUD-843 (the campaign this leaks from), CLOUD-1137 (the census globbing one directory — the counting half of this blindness, where this is the instance), CLOUD-1132 (the two-shapes rule that governs the third file), CLOUD-1208 (which added the third bench/ helper and reverted it), CLOUD-418 (a new gate never shown to fail).
Acceptance
git ls-files '*.py' -- bench/ | wc -l**is **0— the 596 lines underbench/are gone, withacquisition-benchstill producing the same four-term report andacquisition_metric.rsstill asserting the task setsBENCH_METRIC=acquisition-wall-clockover whatever replaced it.bench/gates/RESULTS.mdeither loses its claim to a live generator or leaves with it, and the three source comments citing it still resolve.mise-tasks/replay-pointers.pyis retired under the two-shapes rule withtests/replay.batsandmise-tasks/replay.sh's call site moving in the same change, or it is left whole — not half-ported.- Both withdrawals above stay recorded, so neither the
.pyexclusion is re-proposed as a hole a third time nor the scope limit re-read as a licence a second.
|
Warning Review limit reachedNext included review available in 13 minutes. View limit detailsLimit details: You’ve used the included review currently available. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. Review configuration: ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Free Run ID: 📒 Files selected for processing (16)
Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/settings/billing. Comment |
ed9e3de to
f59fd6a
Compare
… what each rule cost `batten-check` is 465s of the `ci` job's 1327s and emits two lines while it runs. Nothing reported a per-rule duration and every `command` child's streams are `Stdio::null()`, so the largest item in this repository's CI was unattributable from its own output — two sessions guessed at it and both were wrong, by 4.5x and by 56x, and the answer only came out of a scratch worktree with the ruleset hand-edited. Two changes, and the census is first because it is what proves the second. THE CENSUS. `run` times each `run_rule` dispatch and takes a delta over two new process-global read counters, on `DOCUMENTS_ACQUIRED`'s idiom and for its reason: a counter beats a clock where the thing being counted is well inside the noise of a process start, and a delta needs no accumulator threaded through nine read sites. Sound because the loop is serial, which is a measured verdict rather than an accident. Rendered on the `-vv` rung and nowhere else: a duration is not byte-stable, so it must never reach `-J`, a pointer line or stdout (§6). `RuleCost` hand-writes `PartialEq` to skip the clock, so `Scan`'s derived equality stays the byte-stability property it was and does not quietly become timing-dependent. THE MATCHER. `forbid_in_file` took one path and was called once per file, and it called `Matcher::for_rule` — so `Regex::new` ran once per (rule, file) pair, ~3,300 compilations of 17 expressions over this repository's own ruleset. The comment on that call said the compile had been hoisted out of the LINE loop because "`Regex::new` is the expensive half"; it was right about the cost and stopped one loop short. The expression is a property of the row, so `forbid_in_files` takes the matched set and compiles it where the row is. A config fault now also refuses before any file is read rather than after the first. Findings are unchanged: same rules, same order, same pointers, same keyed identities. `matched` is already ordered and the outer sort is untouched. `run_rules` bought its line back by dropping a duplicate — `base_ref` and `config_from` bound the identical `overrides.config_from.as_deref()` under two names, so one value had two spellings in one function. Refs: CLOUD-1217
…per line The census this branch just landed named the pole on its first run, and it was neither of the things this work had been chasing. Two Rego modules, both with the same defect: a FUNCTION that scans the whole file, bound inside a per-line loop, so the scan ran once per line. shell-hygiene 28252ms -> 1919ms (14.4x) remedy-authorship 10984ms -> 1842ms (6.0x) batten enforce 65.25s -> 37.22s `sibling-resolves.rego`'s arm 3 called `dir_vars(path)` — a scan of every line — from `var_names(path, line)`, which `constructed` evaluates per line. O(lines**2) per selected file, over ~140 shell programs. `remedy-authorship.rego`'s `stderr_block` bound `openers_for(path)` the same way and then `closes_to_stderr` scanned again on top. Both are now partial rules keyed by path. A partial rule is evaluated once and indexed; a function is called. Nothing else about either predicate moved, and that is what the modules' own cases check — 262 pass, and `batten enforce`'s whole output is byte-identical before and after: same findings, same order, same pointers. WHAT THIS CORRECTS. The row this branch carries was filed blaming `no-secrets` (3%), then rewritten blaming `forbid` and `ratchet` read amplification (82%). The second attribution came from a kind bisect in a scratch worktree whose arms were not paired — arm one paid a cold page cache and the rest did not, so warmth was read as a rule-kind cost. The census measures directly and says every `forbid` and `ratchet` row together is ~150ms: `no-new-ignores` reads 400 files and 15.9MB in 32ms. The 7.3x read amplification is real and it is worth milliseconds. The matcher hoist in the previous commit stands on its own terms — ~3,300 needless `Regex::new` calls is still wrong — but it is not where the time was, and nothing here should be quoted as if it were. Refs: CLOUD-1217
The census landed in 99420ba with no case under it, which makes it a log rather than a mechanism — non-negotiable rule 2 refuses exactly that. Three cases, in their own binary for `document_read_count.rs`'s reason: the counters are process-global and read as a delta, so a sibling case reading a file in the same process would race them under a threading harness. every_rule_gets_one_census_row_in_declaration_order a_rule_reports_one_read_per_file_its_glob_selected the_census_measures_the_run_rather_than_identifying_it SHOWN ABLE TO FAIL rather than asserted to be. Dropping `count_read` from `forbid_in_files` and re-running gives `left: 0, right: 3` on the second case — 2 passed, 1 failed — so the count tracks the engine rather than a constant. Each case also names its own mutation in a comment, and the second carries an anti-vacuity arm that widens the glob and watches both counts move. The third case is the one that is easy to skip and should not be: `Scan` derives `PartialEq` and two runs over one unchanged tree are the same scan, which is what byte-stability rests on. `RuleCost` therefore compares on its deterministic half and skips `elapsed`; deriving it would make scan equality differ by a nanosecond and be quietly false. Also corrects `Scan::costs`'s own doc: it says every rule, not every EVALUATED rule, because a rule whose glob selected nothing still earns a row reporting zero. "This rule cost nothing" and "this rule is missing from the report" are different answers, and the first case pins that. Refs: CLOUD-1217
…right to refuse it
`batten semver check` refused the branch with `constructible_struct_adds_field`:
`costs` was a `pub` field added to `pub struct Scan`, and a consumer constructing
`Scan { .. }` cannot keep compiling across that. The gate was right about the API,
and reading it settled the design too.
A census is a MEASUREMENT ABOUT a run, not part of the run's value. The tell was
already in the diff: keeping it on `Scan` forced a hand-written `PartialEq` that
skipped `elapsed`, so scan equality would not become timing-dependent — and a
value type that has to lie about one of its fields to stay comparable is carrying
something that is not its own. `RuleCost` derives `PartialEq` now, because nothing
compares a census for identity.
So it moves to `RULE_COSTS`, beside the two counters it is assembled from and on
the idiom the module already cites for them. The one thing a per-rule LIST owes
over those counters: they are monotonic and read as a delta, and a list read that
way would hand a caller the previous run's rows as well — so `run` CLEARS the
store before it fills it, and a reader gets the run that just finished.
`the_census_describes_the_last_run_rather_than_accumulating` is that property,
replacing the `PartialEq` case the move made moot.
Declaring the break instead was the alternative and it would have been the wrong
trade: a version bump spent so a diagnostic could live in a value type it does not
belong in. The public surface is unchanged; `rules::rule_costs()` is new, and
adding a function is not a break.
Refs: CLOUD-1217
…grams already declare `[tasks."test:bats"]` declared `depends = ["doctor --no-targets"]` — the submodule checkout — and nothing else, so it NEVER built the binary. Every gate it drives that needs `batten` paid `cargo run` process startup per case, and worse, whichever artifact happened to be lying in `target/` decided what the suite tested. `tests/helpers.bash:105` asserted the opposite in a comment for its whole life: "`test:bats` builds the DEBUG binary", true only transitively via whatever ran before. That is CLOUD-592's and CLOUD-699's stale-binary class from both ends, and closing it is this change's case. The comment is corrected in the same commit rather than left to read as though it were describing the new behaviour all along. `hooks-wiring-check.sh:212-214` insists its gate judge "the WORKING TREE's engine and config together, which is the pair that ships". A freshly built binary is therefore the CORRECTNESS precondition for the injection, not an optimisation detail — which is why the build is guarded and the export is guarded separately. ## Why it lives here and only here `$BATTEN_BIN` is honoured by seven `mise-tasks/` programs and by the Rust tier at `common/mod.rs:154`; `hooks-wiring-check.sh:219` declares the same seam under its own name and only 2 of its 36 cases ever set it. Exporting both from the ungoverned task reaches every already-honouring program with ZERO governed-file edits — `V-SHELL-RULE-EDITED` refuses touching any of them, one route, no override, which is the whole reason the change has this shape. Inline rather than a `depends`, for two reasons already recorded in this body: a `depends` re-runs in whatever process invokes the task, and when that is hk's step it is a CHILD mise process — CLOUD-220's race — and a build in the DAG would contend with the cargo chain for the target-dir lock `hk.pkl` serialises. `build:release` is the wrong task besides: it produces the RELEASE binary for the mediated-call hot path, while `helpers.bash:119-134` and `common/mod.rs:154` both resolve a debug one. Guarded explicitly, because this body runs under `/bin/sh` with no `set -e`. An unguarded build would leave the suite running against a stale artifact and reporting green — the exact failure the change exists to remove. The perf bonus is the row's ~23s reachable figure, NOT the 135.1s it was filed on; that correction is already on the issue and is not re-argued here. The measurement the row's acceptance demands is reported separately. Refs: CLOUD-1198, CLOUD-592, CLOUD-699, CLOUD-1160
…the fourth
The shell suite has had an instrument since CLOUD-386 — `mise run suite-bench`,
`bench/suites/RESULTS.md`, 144 suites, 1440.9s serial. The Rust suite has had
none, so `mise run test:cargo` emits ONE duration for a step that is four costs
and the per-step receipt makes even that unobservable on a hit. Every claim about
what that suite costs, this bundle's siblings included, was a hand measurement
somebody took once.
total wall = build + execute + <residue>
## The fourth term is the whole reason this exists
Measured warm, the suite is 231s wall against a 127.0s `Summary` and a 5.9s
freshness check, leaving ~98s — 42% of the loop — that nothing in this repository
could attribute. CLOUD-1208 has now been wrong about that residue twice:
1. Filed quoting 1376s wall and "~90% is compile and link", both from guessing
when the run started and ended. Wrong by 4.5x, corrected by `stat`.
2. Then quoting the residue as nextest's per-binary list phase — reasoned from
how nextest works, never measured. Wrong by 56x: a zero-match filter run
pays the freshness check AND the full enumeration and totals 1.75s.
Both were confident, both were plausible, and a harness printing
`list phase: 97.7s` would have shipped the second as fact. So the report prints
the residue as a subtraction and prints what it is NOT, every run, rather than
leaving that in a comment. A report emitting only the `Summary` would declare a
127s suite that takes 231s.
## A sensor, never a gate
A duration ceiling is met by deleting assertions, which is strictly worse than a
slow suite because the result also has to be maintained — `[tasks.coverage]`'s
recorded argument, and non-negotiable rule 2's "a log without a gate is sensor
only". No threshold, absent from `verify` and from `final`'s `needs:`, absent from
`$CI_REQUIRED_CHECKS`.
## Shape
A `bench/rust/sweep.py` driven by a single-line task, copying
`[tasks.acquisition-bench]` exactly, because that is the shape left available:
`V-SHELL-RULE-ADDED` refuses adding a `mise-tasks/*.sh` program, and a helper
under a task directory would publish a second entry point running the sweep with
no `BENCH_METRIC` set. A single-line body also leaves
`inline-task-bodies-not-growing` flat.
`crates/batten/tests/suite_metric.rs` is `acquisition_metric.rs`'s assertion for
the third series, and the third is what turns a pair into a rule: it pins that the
stamp is set, that it is set on the task that actually runs the harness, and that
the three series are pairwise distinct — asserted against the sibling's LIVE stamp
rather than a second literal, since the claim is about the two tasks disagreeing.
## Found by running it
The first real run failed with "nextest printed no Summary line" over a suite that
had just reported `Summary [ 143.437s] 3194 tests run`: nextest colours that line,
so every `\s*` in the pattern was looking at an SGR escape. Fixed with both halves
— `--color never` so the common path is clean, and a strip for whatever still
arrives — and the fix is proven against the captured bytes rather than by a
re-run: raw fails, stripped yields `143.437 / 3194 / 122`. Losing the Summary
silently would have reported the residue as the whole non-build cost, which is
precisely the mislabelling this row is about.
The scheduled workflow §3 asks for is NOT in this commit: `.github/workflows/**`
is in `protected`, `V-PROTECTED-MUTATION` declares no override route, and the
engine's hatch is not settable from inside a tool call. Raised rather than worked
around.
Refs: CLOUD-1208, CLOUD-365, CLOUD-111, CLOUD-386, CLOUD-935
… measured
`[profile.dev]` is one dial for two populations: the library under iteration, and
~118 integration test targets nobody attaches a debugger to. Nothing counted the
second population's bytes until CLOUD-766 hit a full disk.
MEASURED on one container, same `mise run test:cargo` both ways, counting
`target/debug/deps`' extension-less linked binaries — the population
`crates/batten/src/prune.rs:262-269` already reads:
debug = 1 122 artifacts 15.11 GB mean 123.8 MB target/debug 19.18 GB
debug = 0 123 artifacts 1.60 GB mean 13.0 MB target/debug 4.14 GB
**9.4x off the linked artifacts and 4.6x off the whole tree**, suite green both
ways (3194 tests). CLOUD-1211 was refined against "an arm that halves 14.1 GB";
this is well past that, which is why the byte delta is a SEPARATE acceptance test
from the time delta — an arm this large is worth adopting whatever its wall clock
does, and the wall clock is reported against CLOUD-1208's null rather than quoted
from a hand timing.
## Two corrections to the row, both recorded in the manifest
`debug = 1` was ALREADY a reduction: cargo's dev default is `2`, so
line-tables-only was the saving the profile's comment was written about and that
comment was correct as written. The remaining headroom was 1 → 0, not 2 → 0.
And §3 asked for `debug` "scoped to test targets", which cargo cannot express:
profiles are per-PACKAGE, not per-target-kind. `[profile.dev.package."*"]` is the
nearest expressible split — dependencies stripped, workspace code keeping its
symbols — and is the arm to reach for if a backtrace ever needs them back.
The linker claim this row was FILED on stays withdrawn and stays recorded so it is
not re-proposed: rust-lld has been the default on this host triple since Rust
1.90, this repository pins 1.97.1, and `readelf -p .comment` reports
`Linker: LLD 22.1.6`.
## The gate
`crates/batten/tests/dev_profile.rs`. The regression it catches is silent by
construction — restoring `debug = 1`, or dropping the key so it falls back to
cargo's `2`, costs nothing a test run can observe. The only symptom is
`target/debug` growing back by an order of magnitude until a session runs out of
disk, arriving as an unrelated rustc IO error inside somebody else's test run,
which is the misattribution CLOUD-766 records.
So the absent-key case is asserted directly rather than left to the happy path: a
defaulting lookup would read a missing `debug` as satisfied and pass over cargo's
`2`, and that anti-vacuity case is the one that would actually catch the drift.
`[profile.release]` and `[profile.dist]` are pinned untouched, since a test-loop
change must not reach the shipped artifact's profile.
Refs: CLOUD-1211, CLOUD-1208, CLOUD-766, CLOUD-1210
…sable null Three defects, every one found by RUNNING the harness rather than by reading it — `.claude/rules/policy-modules.md`'s second-tier argument arriving on a bench program. The first shipped in c3771a2; the other two are what its first real readings exposed. ## It would have retired this row's own subject With `cargo nextest` as the total, the residue came out at **-1.0s and -0.1s** across two arms — a tidy zero, over the exact suite whose residue CLOUD-1208 measured at 97.7s and built this instrument to explain. Both readings are correct and they answer different questions. The row's 231s was `mise run test:filter`, so the unattributed cost is in the TASK — mise startup, the task graph, `step-receipt` — and never was inside nextest at all. A harness measuring nextest would have reported a residue-free suite and closed the row's subject as solved. That is the THIRD wrong attribution in this row's history and the first one the instrument produced itself, which is the argument for the instrument rather than against it: the two before it took an ad-hoc experiment nobody was obliged to run, and this one took one run of a committed program. So the total is `mise run test:cargo`, and `Summary` is parsed from that same invocation — taking `execute` from a separate run is what made the subtraction go negative, since two runs are not one run. `BATTEN_STEP_RECEIPT_BYPASS=1` rides along, which is the row's own "the per-step receipt makes even that unobservable on a hit" acted on rather than restated: a hit skips the step, so an arm measuring one would publish a cache lookup as the suite's cost. ## The null was 0.750, which is not a null arm=0 execute=123.6s total=124.6s arm=1 execute= 92.5s total= 93.4s ratio=null0 value=0.750 An instrument that cannot distinguish an effect below 25% cannot read any delta a sibling row would quote against it. The arms were not identical: the first warms the OS page cache over 124 test binaries and the second reads them back. `perf-pair`'s consecutive-arm null works because its arms are milliseconds over two committed fixtures; at this scale the first run IS the confound. One discarded warmup arm, three measured. ## And it parsed nothing at all The very first run failed with "nextest printed no Summary line" over a suite that had just printed `Summary [ 143.437s] 3194 tests run`: nextest colours that line, so every `\s*` in the pattern was looking at an SGR escape. `--color never` for the common path and a strip for whatever still arrives — losing the Summary silently would report the residue as the whole non-build cost, which is the mislabelling this row exists to prevent. Proven against the captured bytes rather than by a re-run: raw fails, stripped yields `143.437 / 3194 / 122`. Refs: CLOUD-1208
…th quoting
arm=0 build=1.1s execute=101.3s total=102.5s residue=0.1s
arm=1 build=1.1s execute= 97.6s total= 98.8s residue=0.1s
arm=2 build=1.1s execute= 98.6s total= 99.9s residue=0.2s
null-spread low=0.963 high=1.012 pairs=2
The null is **0.963–1.012** where the un-warmed harness gave 0.750, so the
instrument can now read a delta a sibling row would quote instead of swamping it.
It brackets `perf-compare`'s own 0.966–1.102 closely, which is the first evidence
that suite wall clock behaves like the invocation series once the page cache is
not the independent variable.
## CLOUD-1208's 97.7s residue does not reproduce, and that is a result
`total` is `mise run test:cargo` — the task, the same thing the row measured at
231s — and the residue is **0.1-0.2s across three arms**. Nothing is unattributed.
The row measured 231s wall against a 127.0s `Summary` at `debug = 1`. This tree is
`debug = 0`, where the same suite is ~99s against ~99s of `Summary`. The hypothesis
that fits both readings is that the residue was the OS faulting in 122 binaries of
~124 MB each, and that at ~13 MB each it collapses — which would mean CLOUD-1211
removed CLOUD-1208's subject rather than merely sitting beside it.
**Stated as a hypothesis, not a finding.** It is the fourth attribution attempt in
this row's history and the first three were all confident and wrong, so it is not
quoted until the paired `debug = 1` arm is measured with this same harness against
this same null. That arm is CLOUD-1211's time half and is owed anyway.
Refs: CLOUD-1208, CLOUD-1211
…e wrong thing
`03fb6621` set `[profile.dev] debug = 0` across the whole dev profile. That was
the wrong call and this reverts it to the arm CLOUD-1211 §3 actually asked for.
Three arms, each a cold `mise run test:cargo` over a cleared `target/debug`,
counting `target/debug/deps`' extension-less linked binaries:
arm artifacts bytes mean cold wall
debug = 1 (the baseline) 125 15.53 GB 124.3 MB 231s
debug = 0 (the whole profile) 123 1.60 GB 13.0 MB 264s
debug = 1 + package."*" debug = 0 125 6.82 GB 54.5 MB 217s
## Why the biggest number is the rejected one
`debug = 0` is 9.7x and it buys that by dropping debuginfo from the whole dev
profile, batten's own code included — so a panicking test reports a backtrace
with no line numbers. That is the diagnostic a reader needs at exactly the moment
it is gone, and no byte count is worth it. §3 said as much: scope it away from the
test targets, "leaving `[profile.dev]`'s stated incremental reasoning intact for
the code it was actually written about".
Cargo cannot express per-target-kind scoping, and the previous commit recorded
that constraint, named `[profile.dev.package."*"]` as the nearest expressible
split — and then took the blunt instrument anyway. Measured, the split is where
the bytes were: 124.3 MB to 54.5 MB per binary, 2.3x off the tree, with every
`batten` frame keeping its file and line.
## The wall-clock column is withdrawn, not quoted
`03fb6621` floated the residue-is-debuginfo-load hypothesis. It is unsupported and
the numbers lean against it: `debug = 0` was the SLOWEST cold arm of the three.
All three are single unpaired cold runs at different contention, disagreeing in
both directions and sitting far outside CLOUD-1208's measured 0.963-1.012 null —
which is what unpaired readings look like, not a finding.
The paired warm arm that would decide it CANNOT BE TAKEN in this container: at
`debug = 1` the tree leaves 6.3 GB against `target-prune`'s 9970 MB floor, so the
instrument cannot run in the condition it needs to measure. That is CLOUD-766's
exhaustion arriving inside the experiment, and it is recorded rather than worked
around.
So the byte column is the measured result and the time column is recorded only so
nobody re-runs it expecting an answer.
## The gate
`dev_profile.rs` now pins BOTH halves, because either alone is silently
insufficient: workspace `debug = 1` is the half that must not be traded for bytes,
and the `package."*"` override is the half the bytes come from. Dropping the
override is invisible until a session runs out of disk. The anti-vacuity case is
kept and re-aimed — it pins that `declared_debug` PANICS on an absent key, since a
defaulting lookup would read a dropped setting as satisfied.
Refs: CLOUD-1211, CLOUD-766, CLOUD-1208
… 100.2s to 48.6s
mise run test:cargo, warm 100.189s -> 48.581s 3201 tests, all passing
batten hook, real config 325ms -> 70ms (release is 48ms)
**2.06x on the whole suite**, against CLOUD-1208's measured 0.963-1.012 null. One
line: `[profile.dev.package."*"] opt-level = 2`.
## What the per-case census actually said, once it was taken cleanly
CLOUD-1223 was filed against a ranking taken while the box was busy, and it was
wrong about where the cost is. Re-measured idle, the four `cli.rs` cases it names
are inflated 4.6-9.7x by contention alone, and the binaries that are NOT inflated
— `board_receipts` 43.8s, `pipeline_shapes` 25.0s, `mediated_verbs` 20.1s,
`connector_verbs` 3 cases at 2.3s each — are uniformly ~1-3s per case with no
artifact in the number.
What every one of them has in common is a fixture carrying a REAL config, and
therefore a `batten hook` invocation that loads the committed ruleset and compiles
~20 Rego modules. Measured, that invocation is **325ms on the debug binary against
48ms on release** — so the suite was paying a 6.8x tax per policy evaluation, and
`board_receipts` alone makes ~135 of them.
The tax is not in this repository's code. It is dependency code — Rego compilation
and evaluation — running unoptimised because `[profile.dev]` never said otherwise.
Optimising the closure while workspace code stays unoptimised recovers most of it
and costs the edit-test loop nothing, because the code under iteration is the half
still built for fast rebuilds.
## Three mechanisms this replaces, each withdrawn on measurement
Recorded so none is proposed again: **share one `enforce` run** across the four
`cli.rs` cases — impossible, each writes different content into its own fixture, so
they are different computations. **Stub the spawning rules' binaries** — impossible,
`claim-not-raced` and both `sbom-ntia` rows glob `Cargo.lock` and
`mise-tasks/claim-race-check.sh`, which no fixture contains, so they never spawn
there at all. **Rewrite the 20s `process_group` case** — pointless, it is a
wall-clock wait on a real drain deadline, so it occupies a worker without
competing for CPU and fixing it buys nothing at four workers.
All three were read off durations. This one was read off the binary.
## The cost, stated rather than buried
A cold dependency build goes 217s -> 366s. Paid ONCE and then cached: dependencies
recompile only when they change or this profile does, and CI already carries
`Swatinem/rust-cache`. +149s once against -51.6s every run, and the warm loop pays
none of it.
Refs: CLOUD-1223, CLOUD-1211, CLOUD-1208, CLOUD-352
…duce
Removes `bench/rust/sweep.py`, `bench/rust/RESULTS.md`, `[tasks.suite-bench-rust]`,
`crates/batten/tests/suite_metric.rs` and the `.prettierignore` entry, all added
earlier on this branch.
## CLOUD-1208's subject does not exist
The row was built on a 42% unattributed residue: 231s wall against a 127.0s
`Summary` and a 5.9s freshness check, leaving 97.7s nothing could explain.
Measured at task level with the step receipt bypassed, across three arms:
arm=0 build=1.1s execute=101.3s total=102.5s residue=0.1s
arm=1 build=1.1s execute= 97.6s total= 98.8s residue=0.1s
arm=2 build=1.1s execute= 98.6s total= 99.9s residue=0.2s
**The residue is 0.1-0.2s.** The 97.7s came from reading log-file mtimes at
`debug = 1`, not from an instrument. So the row is answered by the measurement
rather than by a harness, and that answer is on the row.
## The instrument found nothing, and that is the honest reason to remove it
nextest already emits three of the four terms — `Summary` is the execute term and
the total, `--no-run` is the build term, per-case durations come out of the same
output. What the harness added was a subtraction that comes out zero and a null
that is two runs and a division.
Every result this branch actually rests on came from somewhere else: the per-case
distribution from parsing nextest's log, the 325ms-vs-48ms hook cost from timing
the binary directly, the 2.06x from comparing two `Summary` lines, the artifact
deltas from walking `target/debug`.
## And it was a language this repository does not pin
`python` is absent from `[tools]`, so `#!/usr/bin/env python3` ran under whatever
the host happened to have — against the rule that CI, hk and a developer's shell
run byte-identical commands. The scheduled workflow §3 asked for was never written
(`.github/workflows/**` is protected and `V-PROTECTED-MUTATION` declares no
override route), and its `install_args` named `rust cargo-nextest` and no python,
so the harness would not have run on a runner at all.
That hole is wider than this file and is CLOUD-1229's.
`suite_metric.rs` goes with it: 119 lines whose whole subject was that the removed
task set an environment variable.
What survives from the row is what it was really for — the residue is measured and
does not reproduce, recorded where the next reader will find it.
Refs: CLOUD-1208, CLOUD-1229
…chet's scope is shell, the rule is no spaghetti
Two `.py` helpers had grown under `bench/`, outside the retirement ratchet's
glob, and the second cited the first. `[tasks.acquisition-bench]`'s own comment
recorded that the shape was forced — `shell-retirement` refuses ADDING an
authored shell rule at `deny`, one route, no `bypass_env` — so the measurement
went to the one language the ratchet does not watch. A second author read that
comment, followed the precedent for the identical stated reason, and added a
third helper (reverted on CLOUD-1208). That is a growth path with a citation
trail, and this closes it.
A ratchet's subject is authored shell because that is what it was built to
retire. That is a statement about its reach, never a licence for what sits
beside it.
bench/gates/classify.py 269 lines, invoked by nothing — deleted.
bench/acquisition/sweep.py 327 lines, one caller — ported.
`bench/gates/RESULTS.md` stays: three source comments cite it as landed
measurement evidence, so the artifact is load-bearing where its generator was
not. Its header stops claiming a live generator and says what it now is — a
frozen census taken 2026-08-23, not refreshed.
The sweep moves into `crates/batten/src/perf.rs`, which is where the paired
measurement already lives and is the module `policy/spawn-adapters.rego` already
places for exactly this class: a harness whose whole subject is what an EXTERNAL
process costs, so the spawns are the thing rather than an implementation of it.
Sharing that module is also what stops a second percentile convention, a second
record shape and a second hyperfine invocation from existing. The unpinned
interpreter goes with the files: `python` was never in `[tools]`, so every one
ran under whatever the host happened to have while `lock-complete` held every
declared tool to three platforms.
AN EXAMPLE TARGET RATHER THAN A VERB, and that is a refusal honoured rather than
a preference. It was written as `perf acquire` first, on `perf pair`'s
precedent. `crates/batten/tests/pointer_only.rs` sweeps EVERY leaf verb over a
bare fixture corpus and refuses one that exits 3 — "it failed internally, so
what it did not emit proves nothing" — and a sweep with no benchmark runner and
no built binary to time has could-not-look as its only honest answer there.
`perf pair` survives that sweep because it has a real SKIP predicate; there is
no analogue here, and inventing one to satisfy a census is the false green these
gates exist to catch. So the harness is a target the command surface does not
carry, and it spawns nothing.
Tests, in the two tiers `.claude/rules/policy-modules.md` asks for:
perf.rs's own module the sweep-point parse and its refusals, the rendered
reading byte-for-byte, and an anti-vacuity case that an
empty null set prints no spread — the fold's identities
would otherwise publish `low=inf high=-inf` as a
measured width.
acquisition_sweep.rs over the compiled binary, that the GENERATED FIXTURE is
a tree the engine accepts. The load-bearing case seeds
the module's sentinel into one declared document and
asserts the finding names both the row and that
document, because a fixture whose `documents` array the
engine never reads still draws a tidy curve — the
defect `weaver` shipped in the field, exit 0 over a
knowingly-broken registry.
Two gates caught defects on the way, both real: `document_facts` refused prose
naming a consumer artifact inside `crates/batten` (non-negotiable rule 1), and
clippy found a `redundant_closure` in `dev_profile.rs` that had already been
pushed.
`mise-tasks/replay-pointers.py` is the last one and is not reachable from here:
its caller is governed, so it can only leave by retiring the `replay` gate whole
under the two-shapes rule. Filed as CLOUD-1232, Ready, with that shape spelled
out.
Refs: CLOUD-1229, CLOUD-935, CLOUD-1208, CLOUD-843, CLOUD-1132, CLOUD-1232
f59fd6a to
45601ad
Compare
|
❌ The last analysis has failed. |
|
/fast-forward |
…name The escalation knew names and not shapes, so every cross-compilation target tree was outside it entirely. Measured on this container while lapping #775, from a completely cleared `target/`: debug 6429 MB partly reachable (`incremental`, `build`) semver-checks 1548 MB declared by name aarch64-apple-darwin 1378 MB UNREACHABLE release 872 MB UNREACHABLE x86_64-pc-windows-gnu 721 MB UNREACHABLE perf 562 MB declared by name Nothing supersedes those trees, so the retention rule cannot reach them either. They are caches of `incremental`'s exact kind: regrowable, unbounded, and superseded by nothing. WHY THIS IS THE ENGINE'S AND NOT TWO MORE `[[prune.regrowable]]` ROWS. The module header argued that which directories a build tree grows is a fact about the consumer's project, so the list belongs in config. That is true of a NAME and not of a SHAPE, and the distinction is the whole of this change: `semver-checks`, `perf` and `flycheck*` are names somebody chose, so they stay declared. `target/<triple>/` is not — it is what cargo lays down for every `--target`, in every project, and a nested `CARGO_TARGET_DIR` has the identical shape. Recognising that shape compiles in no consumer identifier and matches nothing at all in a project that never cross-compiles, which is what non-negotiable rule 1 is actually about. The header now says so rather than arguing the opposite. The derived roots join the WARM tier and never move the basis, on the file's own existing test: dropping a nested tree makes only *that* build full and leaves the host cargo build warm — the identical reasoning already recorded for `semver-checks` and `perf`. They are taken after the declared pass, and a directory a declared row already removed is skipped, because `remove_dir_all` on an absent path would count a reclaim that freed nothing. SHOWN ABLE TO FAIL (CLOUD-418), because this function ends in `remove_dir_all` and the one case that matters is `target/debug`. A predicate matching it would delete the host build every time the floor was breached and report it as a reclaim. It is excluded twice — structurally (a real `target/debug` holds `deps/`, `build/`, `incremental/`, never a nested profile directory) and by name — and the redundancy is deliberate. Proved rather than asserted: removing the name guard reds `the_host_profile_directories_are_never_taken_as_nested_trees` on the second assertion, and the guard was restored from a backup taken before the mutation. Seven cases: the cross-target recognition, the host exclusion above, a directory with no profile directory inside it, one-level-only so a tree's own `deps/` is not itself a tree, the warm-tier reclaim leaving the basis warm, the cold tier never taking a derived root, and a tree a declared row already took not being counted twice. WHAT THIS DOES NOT FIX, stated rather than implied. The lap still costs 11684 MB and the floor still ratchets up off the worst lap on record — measured this session, a cold lap raised the observed warm floor from 9970 MB to 10997 MB, so the next lap needs ~22.7 GB free at open. This closes 2.1 GB of that gap. `target/release` is deliberately left alone: `perf-gate` rebuilds it partway through the same lap, so reclaiming it would cause a rebuild inside the lap rather than a reclaim from it. Refs: CLOUD-1240, CLOUD-1157, CLOUD-1155, CLOUD-766, CLOUD-418
Closes CLOUD-1217. Closes CLOUD-1198. Closes CLOUD-1208. Closes CLOUD-1211. Closes CLOUD-1223. Closes CLOUD-1229.
Two headline results, both measured rather than argued:
batten enforce(the CI gate tail)mise run test:cargo, warmbatten hook, real config, debug binaryBoth deltas are read against CLOUD-1208's measured repeat-run null of 0.963–1.012.
CLOUD-1217 — two Rego modules re-scanning every line, once per line
sibling-resolves.regoandremedy-authorship.regoeach bound a whole-file scan inside a per-line loop. A Rego function is called; a partial rule is evaluated once and indexed. Both were functions.shell-hygieneremedy-authorshipOutput byte-identical — 8 findings, same order, same pointers — and 262 module cases green.
The instrument came first, and that is the point. This row carried three root causes and the first two were wrong:
no-secrets(wrong by 32x — ripsecrets covers all 833 paths in 2.62s) andforbid/ratchetread amplification (wrong, and from a kind bisect whose arms were not paired). Neither was checkable, becauserun_oncenulls every child's output and no rule kind reported its own duration. So the first commit is a per-rule cost census on the-vvrung. It named the pole on its first run, and the pole was neither thing the row had been chasing.CLOUD-1208 — the Rust suite reports one duration for four costs
The harness this row built was reverted, and the revert is the row's result. It measured four terms separately — build, execute, total, and the residue reported as a residue and never labelled, because two attempts to name that residue were wrong by 4.5x and 56x and a harness printing
list phase: 97.7swould have shipped the second as fact.Running it found three of its own defects: it parsed nothing (nextest colours the
Summaryline), it measured the wrongtotal(cargo nextestrather than the task, which reported a residue-free suite over the exact suite whose residue the row measured at 97.7s), and its null was 0.750 until a warmup arm was added — consecutive arms are not identical when the first faults in 124 binaries.What it then reported was that the row's premise did not reproduce. The 42% residue it was built to attribute was not there once the total was the task rather than nextest, so the instrument was measuring a gap that had already been explained. It kept the number the rest of this PR is read against — the 0.963–1.012 null — and was removed rather than left standing as 303 lines nobody would run again.
CLOUD-1223 — the suite halves, and it was one profile line
Every test judging a real config invokes
batten hook, which loads the committed ruleset and compiles ~20 Rego modules — 325ms on the debug binary against 48ms on release. The suite paid that 6.8x tax per policy evaluation;board_receiptsalone makes ~135 of them. The tax is dependency code running unoptimised, so[profile.dev.package."*"] opt-level = 2recovers it while workspace code stays unoptimised and the edit-test loop pays nothing.Three mechanisms were withdrawn on measurement before this one, all recorded so none is re-proposed: sharing one
enforcerun (the fixtures hold different content, so they are different computations); stubbing the spawning rules' binaries (claim-not-racedand bothsbom-ntiarows globCargo.lockandmise-tasks/claim-race-check.sh, which no fixture contains — they never spawn there); and rewriting the 20sprocess_groupcase (a wall-clock wait that occupies a worker without competing, so it is worth nothing at four workers). All three were read off durations; this one was read off the binary.The ranking that suggested them was taken on a busy box: idle, the four
cli.rscases it named are inflated 4.6–9.7x by contention alone.CLOUD-1211 — dev debuginfo, and the biggest number is the rejected one
debug = 1(baseline)debug = 0, whole profiledebug = 1+ deps0(adopted)The middle arm is 9.7x and was briefly committed, then reverted: it drops debuginfo from the whole dev profile, so a panicking test reports a backtrace with no line numbers — the diagnostic a reader needs at exactly the moment it is gone. The time column is withdrawn, not quoted: three unpaired cold runs disagreeing in both directions, and the paired warm arm that would settle it cannot run here — at
debug = 1the tree leaves 6.3 GB against a 9970 MB floor, which is CLOUD-766's exhaustion arriving inside the experiment.Cargo profiles are per-package, never per-target-kind, so §3's "scoped to test targets" is not expressible; the split above is the nearest thing that preserves its intent.
CLOUD-1198 —
test:batsnever built the binaryIts only dependency was the submodule checkout, so whichever artifact happened to be in
target/decided what the suite tested — CLOUD-592's and CLOUD-699's stale-binary class. Now it builds and exports both spellings of the seam seven programs already honour, with zero governed-file edits (V-SHELL-RULE-EDITEDrefuses every one of them).Acceptance demanded measure-both-ways-and-withdraw-if-noise. Paired over
hooks-wiring-check.bats:Before arms agree to 0.3%, so a 13.7% effect is far outside the instrument's own spread. ~7.7s, not noise — the change stands. The row predicted ~23s reachable; measured is ~7.7s, so its own estimate was ~3x high.
CLOUD-1229 — 596 lines of Python retired out of
bench/Two
.pyhelpers had grown underbench/, outside the retirement ratchet's glob, and the second cited the first.[tasks.acquisition-bench]'s own comment recorded that the shape was forced —shell-retirementrefuses ADDING an authored shell rule atdeny, one route, nobypass_env— so the measurement went to the one language the ratchet does not watch. This session read that comment, followed the precedent for the identical stated reason, and added a third (the CLOUD-1208 harness above, since reverted). That is a growth path with a citation trail, and this closes it.A ratchet's subject is authored shell because that is what it was built to retire. That is a statement about its reach, never a licence for what sits beside it.
bench/gates/classify.pybench/acquisition/sweep.pybench/gates/RESULTS.mdstays:facts.rs:698,rules.rs:1965andgit_facts.rs:5cite it as landed measurement evidence, so the artifact is load-bearing where its generator was not. Its header stops claiming a live generator and says what it now is.The sweep moves into
crates/batten/src/perf.rs— where the paired measurement already lives, and the modulepolicy/spawn-adapters.regoalready places for exactly this class: a harness whose whole subject is what an EXTERNAL process costs, so the spawns are the thing rather than an implementation of it. Sharing it is also what stops a second percentile convention, a second record shape and a second hyperfine invocation from existing. The unpinned interpreter goes with the files:pythonwas never in[tools], so every helper ran under whatever the host had whilelock-completeheld every declared tool to three platforms.An example target rather than a verb, and that is a refusal honoured rather than a preference. It was written as
perf acquirefirst, onperf pair's precedent.tests/pointer_only.rssweeps every leaf verb over a bare fixture corpus and refuses one that exits 3 — "it failed internally, so what it did not emit proves nothing" — and a sweep with no benchmark runner and no built binary to time has could-not-look as its only honest answer there.perf pairsurvives that sweep because it has a real SKIP predicate; there is no analogue here, and inventing one to satisfy a census is the false green these gates exist to catch.Run end to end after the port, exit 0, same four line kinds as the predecessor:
Two tiers, per
.claude/rules/policy-modules.md.perf.rs's own module pins the parse, its refusals, and the rendered reading byte-for-byte — plus an anti-vacuity case that an empty null set prints no spread, since the fold's identities would otherwise publishlow=inf high=-infas a measured width.tests/acquisition_sweep.rsruns over the compiled binary and asserts the thing no unit case can: that the generated fixture is a tree the engine accepts. Its load-bearing case seeds the module's sentinel into one declared document and requires the finding to name both the row and that document — because a fixture whosedocumentsarray the engine never reads still draws a tidy curve, which is the defectweavershipped in the field at exit 0 over a knowingly-broken registry.Two gates caught defects during this work, both real:
document_factsrefused prose naming a consumer artifact insidecrates/batten(non-negotiable rule 1), and clippy found aredundant_closureindev_profile.rsthat had already been pushed.Not in scope
No predicate weakens: no glob narrows, no rule retires, no severity moves, no case is deleted or merged.
Two things are raised rather than worked around.
CLOUD-1208's §3 asks for a scheduled workflow beside
coverage.yml..github/workflows/**is inprotectedandV-PROTECTED-MUTATIONdeclares no override route, so it cannot be written from inside a tool call. Everything else in the row landed without it.mise-tasks/replay-pointers.pyis the last tracked.pyand is not reachable from here: its callermise-tasks/replay.shis governed, so it can only leave by retiring thereplaygate whole under the two-shapes rule — program, suite,$MUTANT_GATESrow, oneconservesarm per deleted path. Filed as CLOUD-1232, Ready and in Todo, with that shape spelled out. Bundling a retirement-shaped change with two deletions that need no such shape is what would strand both.