docs(architecture): align tests with compounding capability

CLAUDE.md

@@ -80,7 +80,9 @@ Headline entrypoints: `runAgentTask`/`runAgentTaskStream` (`src/run.ts`), the mu
 
 ## Self-improvement state & discipline (.evolve/ + the gate)
 
-This repo is the empirical home of the RSI/learning-flywheel thesis, but **mechanism is not evidence**. The binding question is the **gate**: *does any non-blind topology beat blind compute at EQUAL k, under a deployable (non-oracle) selector, on a domain with a correctable middle band, at significant n (paired-bootstrap + BH)?*
+The product is an agent system that improves its decisions and learning methods across runs.
+Before choosing experiments or deleting mechanisms, read [architecture.md §0.5 and §9](./docs/architecture.md) for success criteria and experiment scope.
+Single-run comparisons test specific mechanisms; they do not decide whether capability compounds across projects.
 
 **The live science state — every number, what's proven/disproven, the current goal — lives in `.evolve/current.json` + the `memory/` evidence ledger. Read them; do not mirror them here.** `docs/eval-substrate.md` holds the north star (the RSI runtime + its eval substrate) and the measurement non-negotiables.
 

docs/ANTI_PATTERNS.md

@@ -6,15 +6,12 @@ These are repo-level failure modes that have already cost time or produced
 misleading confidence. If a proposal repeats one, stop and ask what proof would
 make the work legitimate.
 
-## Mechanism Ahead Of The Gate
+## Mechanism Without A Decisive Test
 
-Do not build per-branch adaptive sub-agents, learned planners, corpus promotion,
-outer-flywheel machinery, or other high-ceiling mechanisms before a positive
-gate result. Expressiveness was the closed gap; evidence is the open one.
-
-Required proof: a measured non-blind topology beats blind compute at equal k,
-under a deployable selector, on a domain with a correctable middle band, with
-reported discordant pairs and multiple-comparison discipline.
+Build the smallest complete path that can exercise the claimed mechanism before scaling the implementation or experiment.
+Use [architecture.md §9](./architecture.md#9-build-order-and-experiment-scope) to define the comparison and the conditions for rejecting it.
+A single-run steering result does not decide whether learning across projects works.
+Test necessary combinations together, then remove components to identify their contribution.
 
 ## Facade Before Substrate Join
 

docs/architecture-interpretations.md

@@ -178,11 +178,11 @@ The strongest good-faith case: what's wired is the losing half (self-refine) ste
 
 ## 5. Gate A — the decision gate for the recursive-driver layer
 
-Build the adaptive driver **only if** this comes back positive:
+Test within-run steering with this diagnostic:
 
 > On a held-out benchmark, at **equal worker-compute budget** (`k` counts worker ROLLOUTS — each may be a full multi-turn/stateful trajectory, not a single shot), does a **trace + analyst-findings-fed** driver, scored by a **sound non-oracle selector**, beat **blind random@k** selected by that *same* selector — by a statistically significant margin (n large enough for p < 0.05) that **survives test-retest of the selector**?
 
-Until `refine@k-with-findings > random@k at equal compute under a non-oracle selector`, the recursive-driver layer is unjustified overhead and only the minimal honest version (§6) should be built.
+Use [architecture.md §9](./architecture.md#9-build-order-and-experiment-scope) for resource accounting, mechanism activation, and the conditions for rejecting the tested design.
 
 **Measured: cleared at small n, then RETRACTED to a TIE at power (POWER-16).** On
 EnterpriseOps-Gym itsm, depth-steered continuation (analyst-fed, `observe()`) beat blind
@@ -194,7 +194,9 @@ to a tie when powered, and the program pivoted off this anchor (numbers:
 codegen** (HumanEval), **positive on stateful agentic domains** with a correctable
 middle band scored keep-best (EOPS).
 
-**Gate A ≠ project success.** Gate A is the inner GO/NO-GO for *one* component (the within-run driver). The product-success gate is **Gate B** — a positive cross-run score-vs-run slope under a frozen-controller control ([learning-flywheel.md](./learning-flywheel.md)), which is currently **UNMEASURED** (cf. the zero cross-benchmark-transfer admission, §6). A failed Gate A deletes within-run steering; it never bears on Gate B.
+**Gate A tests one mechanism under specified conditions.**
+Product success is **Gate B**: improvement across runs against an unchanged controller ([learning-flywheel.md](./learning-flywheel.md)).
+The historical results above do not establish whether that improvement occurs.
 
 ---
 
@@ -206,7 +208,9 @@ The **minimal honest version** survives every critique and yields the proven mor
 2. **A deployable, non-oracle selector** scoring each ingest on *trace-observable structural* signal — citation coverage, contradiction-lint pass, staleness, novelty-vs-existing-wiki. This is the missing piece that makes best-of-N actually pay, and it is the same build as landing the *selector ≠ judge* firewall.
 3. **The `llm-wiki` maintainer+critic** as the dedup / cite / lint sink (already exists as a skill).
 
-Then run the §5 gate. If a findings-fed driver beats random@k at equal k under that sound selector, the adaptive driver earns its complexity and is built on top — and this surface becomes the first honest validation of the RSI thesis. If not, ship 1+2+3 — agentic RAG with a verifier — and delete the steering machinery with a clear conscience.
+Run the §5 diagnostic to assess within-run steering under those conditions.
+Apply [architecture.md §9](./architecture.md#9-build-order-and-experiment-scope) before retaining, rejecting, or combining the tested mechanisms.
+Test learning across projects separately before claiming that capability compounds.
 
 ---
 

docs/architecture.md

@@ -25,6 +25,10 @@
 Four claims define the system. The sections below are mechanism in service of these;
 if a section drifts from one of these, the claim wins and the section is wrong.
 
+The target is a persistent research and engineering system that completes complex software and produces independently checked research.
+It can revise its methods, tools, and organization, then retain changes that improve later work.
+Recursive self-improvement includes improving how those changes are discovered and tested.
+
 1. **The atom is a decision, not a spawn.** At every level an agent faces the same
    question: given the solution so far, the feedback so far, and the budget left, what
    is the best next move — *keep working · branch · split · get a second opinion · run
@@ -34,12 +38,12 @@ if a section drifts from one of these, the claim wins and the section is wrong.
    AND cheap. Success is **multi-objective**; we do not collapse it to one number until
    forced to. Today every judge returns a single `score` — that is the **gap to close**,
    not the design (§6, §5).
-3. **Each objective carries its own checker — that is what makes this trainable.**
-   *Fastest* is graded by a clock, *most secure* by a scanner, *correct* by the tests.
-   The objective **is** a deployable verifier (§1's *verifier*, distinct from the oracle
-   and the write-only judge). So the loop has honest, cheap signal at every step, on real
-   work, **without an answer key** — that is the gift the multi-objective framing buys,
-   and the reason depth/continuation has something sound to steer on.
+3. **Each objective needs evidence with explicit limits.**
+   Clocks measure elapsed time; tests exercise specified behavior; scanners detect specified security failures.
+   Passing a check establishes only what that check covers.
+   Research can require proof checking, independent replication, or new experiments when no answer key exists.
+   Agents can author working checks, while independent assessment tests the final claims and records unverified assumptions.
+   The quality and cost of that evidence are part of the research problem.
 4. **The improvement that counts is the policy getting better across runs.** Two things
    improve on two clocks (§2). *Within* a run the **solution** climbs (the artifact gets
    better round over round). *Across* runs the **decision policy** climbs — it remembers
@@ -145,8 +149,8 @@ runs — and *that across-run slope is the success criterion* (**Gate B**, defin
 trace-fed driver beat a blind same-compute baseline under a non-oracle selector at equal
 compute* (**Gate A**, defined in
 [architecture-interpretations.md §5](./architecture-interpretations.md)) — is a separate,
-narrower diagnostic; a failed Gate A deletes within-run steering, never the corpus+policy
-product. Live results for both gates: `.evolve/current.json` + the memory ledger.
+narrower diagnostic governed by the experiment scope in §9.
+Live results for both gates: `.evolve/current.json` + the memory ledger.
 
 ---
 
@@ -270,7 +274,10 @@ across benchmarks**. Infra is the cost of entry; transfer is the company.
 
 ---
 
-## 9. Build order (rung discipline — do not skip)
+## 9. Build order and experiment scope
+
+Choose the smallest complete experiment that can test the claimed advantage.
+The mechanisms below are experiment options; an earlier option winning is not a prerequisite for testing a later option.
 
 1. **Atom instance, inference-time.** Driver (`llm-call`, fed by a trace-analyst
    report) steers a worker over k shots; a **selector** picks the answer
@@ -280,13 +287,25 @@ across benchmarks**. Infra is the cost of entry; transfer is the company.
    one-shot artifact structurally cannot exhibit continuation — the rung-0 "steering loses"
    result is bench-specific, not domain-general). Status: see the ledger
    (`.evolve/current.json`).
-2. **Escalate the driver to `sandbox-agent` (auto-research)** — only if rung 1
-   beats compute-matched random.
+2. **Give the driver execution access through `sandbox-agent` (auto-research).**
 3. **GEPA** the driver/analyst `context`+prompts, held-out gated.
 4. **Composition lift** — `fork`/coordinator/nested (driver-of-drivers).
 5. **Cross-benchmark transfer** — one learned controller, many benchmarks. The moat.
 
-Each rung must beat compute-matched random before the next is funded.
+Test combinations when the claimed benefit depends on interactions between delegation, retained knowledge, continued work, or learned decisions.
+Remove components in controlled comparisons to identify their contribution.
+An isolated component losing does not establish that the combination cannot help.
+
+Measure the hardest independently checked work completed, results across resource budgets, and improvement across successive projects.
+Use equal actual resources within each comparison and keep model versions and task difficulty controlled.
+Report learning costs separately and include them in total costs over the declared project horizon.
+
+Reject a mechanism only within the conditions tested.
+The task must exercise the conditions where an advantage is claimed.
+The mechanism must run observably, and the assessment must detect the smallest useful effect.
+Use enough paired cases and a declared stopping rule to support the decision.
+When those conditions hold and the effect is below the useful threshold, reject or simplify that mechanism for those conditions.
+If a test condition is missing, record what remains untested and name the missing condition before further work.
 
 ---
 

docs/learning-flywheel.md

@@ -36,16 +36,14 @@
 > corpus, the gain survives a **frozen-controller control** (re-running an earlier controller shows
 > no slope), it is significant at adequate n (paired-bootstrap + BH), and it is graded by a
 > **deployable checker** — never the answer oracle or the write-only judge. *Multi-objective* is
-> load-bearing: success is a vector (correct · fast · secure · cheap), and **each objective ships
-> its own deployable checker** — tests, a clock, a scanner, a cost meter. That is what gives the
-> flywheel honest, cheap, oracle-free signal on real work at every step (and what depth/continuation
-> steers on). This OUTER-loop slope is THE success criterion. The
+> load-bearing: success is a vector (correct · fast · secure · cheap), with evidence scoped to each objective.
+> Tests, clocks, scanners, and cost meters provide partial measurements; record each check's coverage and unverified assumptions.
+> This OUTER-loop slope is THE success criterion. The
 > within-run "trace+findings-fed controller beats the blind same-compute baseline under a non-oracle
 > selector at **equal compute**" question is a separate, narrower diagnostic — **Gate A**, the
-> GO/NO-GO for building the recursive-driver layer (see [roadmap-rsi.md](./roadmap-rsi.md)). A failed
-> Gate A deletes within-run steering only; it never touches this corpus+controller product. Equal
-> compute (Σ rollouts × turns per arm) is the anti-confound guard in both gates and is **silent on
-> statefulness**: the budget may be spent as one deep trajectory, K shallow attempts, or any mix.
+> comparison for within-run steering, scoped by [architecture.md §9](./architecture.md#9-build-order-and-experiment-scope).
+> Compare actual resource use in both tests, including learning costs over the declared project horizon.
+> The budget may fund one deep trajectory, several shallow attempts, or a mixture.
 
 ## The flywheel
 
@@ -111,9 +109,8 @@ It is real (not vapor) only under three constraints:
    is *why* the integrity rule (judge never feeds steering/selection) is non-negotiable.
 2. **Shared corpus `D`.** Improvements persist and are evidenced *across* levels — a level-1
    gain shows up in the corpus the level-0 runs produced.
-3. **Rung-by-rung earning.** Level *n* is real iff it **measurably lifts level n−1 on `J`**.
-   Recursion is real ∝ rungs earned; skip a rung and you stack noise on noise. (We are at L0
-   with ~0 confirmed signal — so the stack is the north star, built strictly bottom-up.)
+3. **Evidence per learning level.** Level *n* contributes only when it **measurably lifts level n−1 on `J`**.
+   This requirement concerns evidence; construction and comparison follow [architecture.md §9](./architecture.md#9-build-order-and-experiment-scope).
 
 **This subsumes everything in this repo and this design:** the worker, the `f(trace)` steer,
 the controller-as-signatures, GEPA, `meta-harness`, AND the **skill-governor** (which skill to
@@ -229,9 +226,8 @@ A separate final-test partition is required because source labels alone cannot p
 - **Eval economics is the moonshot bottleneck, not controller cleverness.** Build the offline
   corpus/replay so search is affordable. Don't build the optimizer cathedral over a metric
   you can only sample a few hundred times with overlapping CIs.
-- **Prove signal per rung before escalating cost.** random → fixed → `LLM(trace)` →
-  agentic-driver. Each rung must beat *compute-matched* random before the next is justified.
-  Don't jump to the unbounded agentic driver to (expensively) re-derive that more-compute ≈ 0.
+- **Choose a decisive test before escalating cost.**
+  Follow [architecture.md §9](./architecture.md#9-build-order-and-experiment-scope) for complete mechanisms, combinations, resource accounting, and rejection conditions.
 
 ## Honest status (updated 2026-06-10)