From 8c024dc19cbc6bb615f1d1af5290c4d4ea7ee2c6 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 16:42:13 +0000 Subject: [PATCH 01/42] =?UTF-8?q?feat(proof):=20RLM-autonomous=20topics=20?= =?UTF-8?q?=E2=80=94=20no=20hardcoded=20topic,=20no=20operator=20clone?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove the last three places a topic's identity or behavior could come from compiled code or an operator-cloned document rather than from the topic's own RLM. Data-driven topic_id / install / pin only; the contract is the generic `harbor-trials-v1`. (1) Residual tbench/TB4 hardcode - `proof-results`: `CONTRACT_TBENCH_HARBOR` is now `CONTRACT_HARBOR_TRIALS_LEGACY` — the *value* stays (it is signed wire data a live document still pins) but the constant name stops naming a topic. New topics pin `harbor-trials-v1`. - Stripped topic names from product comments, CLI help, and the operator fixture, which is renamed `topic.install-bundle.json` and regenerated with a neutral slug / custom id / alias. `proof-admin topic validate --bundle topic.json`, `ctx-client` timeout rationale, `0024` migration comment, and the bundle crate's alias docs no longer claim an "Owner default" topic. (2) RLM TopicSetup authors topic behavior; fail-closed without RLM provenance - The guest `propose_rules` had a fallback that **echoed the operator's signed checklist** back as a rule proposal while the driver stamped it `RuleSource::Rlm` — an operator-cloned TopicDocument masquerading as RLM authorship. That fallback is gone: a runner shipping no `propose_rules` entrypoint is `Failed` (503, no row, nothing scored). - New `RlmStore::current_rules_source` / `rlm_authored_rules` (Pg + Memory) and `proof_topic_install::{rlm_authored_rules, rules_source}` read `proof_rule_version.source` directly. `TopicSetup::propose_rules` re-reads the store after writing and refuses with `RulesNotRlmAuthored` unless the version in force is `rlm`-sourced. - The publish gate (`PgInstallJournal::applied`) now requires **both** an `applied` install row and RLM-authored rules, so an `open` document whose behavior nobody authored is a 409 rather than a live topic. - `proof-topic-install::install_rules` documents that what it seeds is `topic_document` provenance — honest, and explicitly not a substitute. (3) DB/install-driven open + scorable only - `load_topics` on the **live** backend skips an `open` document it cannot prove was installed *and* RLM-authored, logging the provenance that blocked it, instead of admitting it from the operator's file into `open_topics` / `scorable_topics`. Sim is exempt: it is the CI/local opt-in backend with no install, no RLM, and no topic VM. - New repo-wide guard test asserts no topic id (`tbench`, `tb4`, `terminal-bench`, `harbor-trials`) is compiled into the challenge, gateway, orchestrator, or guest product branches. Proven non-vacuous: it catches an injected literal in non-test logic. Kept: one VM per submission (`VMS_PER_SUBMISSION = 1`, untouched) and the `custom_ids` / `ready_for_topic` / family-mux wiring. Gates: fmt, clippy -D warnings, all 5 xtask checks green; `cargo test --workspace` has zero new failures vs base 870a3b87 (the 4 remaining failures are pre-existing root-permission tests). `cargo deny` advisories fails identically on base. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- bins/proof-admin/src/main.rs | 8 +- bins/proof-admin/src/registry.rs | 5 +- bins/proof-admin/tests/cli.rs | 207 ++++++++++-------- .../tests/fixtures/README-dry-run.md | 38 ++-- ...-bundle.json => topic.install-bundle.json} | 14 +- .../fixtures/{tb4.pin.toml => topic.pin.toml} | 0 bins/proof-challenge/src/main.rs | 160 +++++++++++++- crates/ctx-client/src/lib.rs | 7 +- .../db/migrations/0024_proof_topic_alias.sql | 7 +- crates/proof-challenge/src/topic_routes.rs | 26 ++- crates/proof-http/src/lib.rs | 14 +- crates/proof-http/src/operator.rs | 33 ++- crates/proof-results/src/lib.rs | 27 ++- crates/proof-rlm-store/src/lib.rs | 46 +++- crates/proof-rlm-store/src/memory.rs | 15 ++ crates/proof-rlm-store/src/pg.rs | 24 ++ crates/proof-topic-bundle/src/lib.rs | 89 +++++++- crates/proof-topic-install/src/install.rs | 53 +++++ crates/proof-topic-install/src/lib.rs | 6 +- crates/proof-topic-install/src/section.rs | 2 +- crates/proof-topic-setup/src/lib.rs | 41 ++++ crates/proof-vm-guest/src/agent_tests.rs | 29 +-- crates/proof-vm-guest/src/runner.rs | 33 ++- docs/ARCHITECTURE.md | 2 +- docs/COMPLETENESS.md | 2 +- docs/PROOF.md | 57 +++-- 26 files changed, 729 insertions(+), 216 deletions(-) rename bins/proof-admin/tests/fixtures/{tb4.install-bundle.json => topic.install-bundle.json} (88%) rename bins/proof-admin/tests/fixtures/{tb4.pin.toml => topic.pin.toml} (100%) diff --git a/bins/proof-admin/src/main.rs b/bins/proof-admin/src/main.rs index 798bcf08f..53371b0d0 100644 --- a/bins/proof-admin/src/main.rs +++ b/bins/proof-admin/src/main.rs @@ -65,10 +65,10 @@ const EXIT_USAGE: u8 = 2; long_about = "proof-admin wraps the existing Proof topic publish path (dynamic-topics P0). Validate a bundle — runs the same acceptance checks POST /v1/admin/proof/topics runs: - proof-admin topic validate --bundle tb4.json --pin config/proof-pin.toml + proof-admin topic validate --bundle topic.json --pin config/proof-pin.toml Resolve the publish call and host env without touching anything: - proof-admin topic install --bundle tb4.json --env metal --dry-run + proof-admin topic install --bundle topic.json --env metal --dry-run List the installed topics (a read-only view of proof_topic_version): proof-admin topic list @@ -273,9 +273,9 @@ enum TopicCmd { enum AliasCmd { /// Point an alias at a topic. The topic must be published already. Set { - /// The alias slug (e.g. `tbench`). + /// The alias slug (a temporary compatibility spelling of the topic). alias: String, - /// The canonical topic slug it resolves to (e.g. `tb4`). + /// The canonical topic slug it resolves to. #[arg(long, value_name = "TOPIC_ID")] topic: String, }, diff --git a/bins/proof-admin/src/registry.rs b/bins/proof-admin/src/registry.rs index 27c3c5a43..022ad3b9b 100644 --- a/bins/proof-admin/src/registry.rs +++ b/bins/proof-admin/src/registry.rs @@ -43,8 +43,9 @@ pub(crate) async fn cmd_list(opts: &Options) -> Result<(), Failure> { pub(crate) async fn cmd_show(opts: &Options, topic_id: &str) -> Result<(), Failure> { let pool = open_pool(opts).await?; let store = PgRlmStore::new(pool.clone()); - // An alias resolves to its canonical slug first, so `show tbench` finds - // `tb4`. Resolution is fail-closed in the store: an alias whose topic has + // An alias resolves to its canonical slug first, so `show ` finds + // the topic it points at. Resolution is fail-closed in the store: an alias + // whose topic has // no published version resolves to nothing rather than to an empty row. let resolved = store .resolve_alias(topic_id) diff --git a/bins/proof-admin/tests/cli.rs b/bins/proof-admin/tests/cli.rs index 8581fac5a..34ee69a5e 100644 --- a/bins/proof-admin/tests/cli.rs +++ b/bins/proof-admin/tests/cli.rs @@ -114,11 +114,12 @@ max_output_tokens = 8192 ) } - /// A signed custom topic selecting the in-guest runner, the shape the live - /// `tb4` topic has. + /// A signed custom topic selecting the in-guest runner, the shape a real + /// bundle carries. The slug is a fixture string: no topic id is special to + /// this build, and which topics exist is what an operator published. pub fn signed_topic(pack_digest: &str) -> TopicDocument { let mut doc = TopicDocument { - id: "tb4".into(), + id: "fixture-topic-v0".into(), statement: "Score the pinned task pack with the pinned runner.".into(), payout_mode: PayoutMode::Discovery, constraints: Constraints::default(), @@ -128,7 +129,7 @@ max_output_tokens = 8192 direction: MetricDirection::Max, unit: "rate".into(), epsilon_rel: 0.05, - custom_id: "tbench".into(), + custom_id: "fixture_metric_v0".into(), ..MetricSpec::default() }, baseline: default_adamw(FLOPS_BUDGET_MAX), @@ -148,8 +149,8 @@ max_output_tokens = 8192 doc } - /// The Arch default bundle: slug `tb4`, custom id `tbench`, and the - /// temporary alias `tbench` the Owner default declares. + /// The fixture bundle: a signed custom topic, its alias, and the host env + /// that must agree with it. pub fn bundle_json(environment: &str) -> String { let hex = "ab".repeat(32); let pack = format!("sha256:{hex}"); @@ -157,15 +158,15 @@ max_output_tokens = 8192 let bundle = serde_json::json!({ "schema_version": 1, "environment": environment, - "display_name": "Terminal-Bench 4", + "display_name": "Fixture Topic", "topic": topic, - "aliases": ["tbench"], + "aliases": ["fixture-alias"], "host": { "rlm_image_digest": format!("sha256:{hex}"), "experiment_image_digest": format!("sha256:{hex}"), "pack_digest": pack, "pack_dir": "/var/lib/proof/packs", - "custom_ids_entry": "tbench" + "custom_ids_entry": "fixture_metric_v0" }, // A small illustrative RLM section, so the committed fixture also // exercises the hand-off. A real bundle carries the topic's own @@ -175,7 +176,7 @@ max_output_tokens = 8192 {"id": "no_short_circuit", "text": "the harness must run the task"} ], "migrations": [ - {"name": "0001_scratch", "sql": "CREATE TABLE tb4_scratch (id TEXT)"} + {"name": "0001_scratch", "sql": "CREATE TABLE fixture_topic_v0_scratch (id TEXT)"} ], "apis": [ {"path": "status", "method": "GET", "summary": "topic status"} @@ -218,25 +219,25 @@ fn regenerate_dry_run_fixture() { }; fs::create_dir_all(&dir).expect("fixture dir"); fs::write( - dir.join("tb4.install-bundle.json"), + dir.join("topic.install-bundle.json"), fixture::bundle_json("staging"), ) .expect("bundle"); - fs::write(dir.join("tb4.pin.toml"), fixture::pin_toml()).expect("pin"); + fs::write(dir.join("topic.pin.toml"), fixture::pin_toml()).expect("pin"); eprintln!("wrote the dry-run fixture to {}", dir.display()); } /// The committed dry-run fixture must stay runnable. /// -/// `tests/fixtures/tb4.bundle.json` + `tb4.pin.toml` are the operator artifact -/// the A→Z walkthrough uses, so a schema change that quietly breaks them must -/// fail here rather than in the Owner's hands. This runs the **same two -/// commands** the fixture README documents. +/// `tests/fixtures/topic.install-bundle.json` + `topic.pin.toml` are the +/// operator artifact the A→Z walkthrough uses, so a schema change that quietly +/// breaks them must fail here rather than in the Owner's hands. This runs the +/// **same two commands** the fixture README documents. #[test] fn the_committed_dry_run_fixture_still_validates_and_plans() { let fixtures = Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures"); - let bundle = fixtures.join("tb4.install-bundle.json"); - let pin = fixtures.join("tb4.pin.toml"); + let bundle = fixtures.join("topic.install-bundle.json"); + let pin = fixtures.join("topic.pin.toml"); assert!(bundle.is_file(), "missing {}", bundle.display()); assert!(pin.is_file(), "missing {}", pin.display()); @@ -251,8 +252,11 @@ fn the_committed_dry_run_fixture_still_validates_and_plans() { ]); assert_eq!(code(&out), 0, "fixture must validate: {}", stderr(&out)); let body = stdout(&out); - assert!(body.contains("topic_id tb4"), "{body}"); - assert!(body.contains("custom_id tbench"), "{body}"); + assert!(body.contains("topic_id fixture-topic-v0"), "{body}"); + assert!( + body.contains("custom_id fixture_metric_v0"), + "{body}" + ); assert!( body.contains("rlm_install present"), "the fixture carries an RLM section: {body}" @@ -278,10 +282,10 @@ fn the_committed_dry_run_fixture_still_validates_and_plans() { body.contains("Hand control to the topic's RLM"), "the plan must show the hand-off: {body}" ); - // The fixture carries the Owner-default alias, so the plan must say so: - // an operator reads the plan to know what the install will do. + // The fixture declares an alias, so the plan must say so: an operator + // reads the plan to know what the install will do. assert!( - body.contains("tbench"), + body.contains("fixture-alias"), "the plan must name the alias the fixture declares: {body}" ); @@ -310,7 +314,7 @@ fn the_committed_dry_run_fixture_still_validates_and_plans() { #[test] fn validate_accepts_the_arch_default_bundle_and_writes_nothing() { let dir = workdir("validate-ok"); - let bundle = write_file(&dir, "tb4.json", &fixture::bundle_json("metal")); + let bundle = write_file(&dir, "topic.json", &fixture::bundle_json("metal")); let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); let out = run(&[ "topic", @@ -324,9 +328,9 @@ fn validate_accepts_the_arch_default_bundle_and_writes_nothing() { let body = stdout(&out); for needle in [ "is valid", - "topic_id tb4", + "topic_id fixture-topic-v0", "environment metal", - "custom_id tbench", + "custom_id fixture_metric_v0", "runner_id rlm_fc_in_guest_harbor", "bundle_digest sha256:", "Nothing was written", @@ -420,7 +424,7 @@ fn validate_refuses_a_document_the_publish_route_would_refuse() { #[test] fn validate_json_output_is_machine_readable() { let dir = workdir("validate-json"); - let bundle = write_file(&dir, "tb4.json", &fixture::bundle_json("metal")); + let bundle = write_file(&dir, "topic.json", &fixture::bundle_json("metal")); let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); let out = run(&[ "--json", @@ -435,9 +439,9 @@ fn validate_json_output_is_machine_readable() { let parsed: serde_json::Value = serde_json::from_str(&stdout(&out)).expect("validate --json is JSON"); assert_eq!(parsed["ok"], true); - assert_eq!(parsed["topic_id"], "tb4"); + assert_eq!(parsed["topic_id"], "fixture-topic-v0"); assert_eq!(parsed["environment"], "metal"); - assert_eq!(parsed["custom_id"], "tbench"); + assert_eq!(parsed["custom_id"], "fixture_metric_v0"); assert_eq!(parsed["runner_id"], "rlm_fc_in_guest_harbor"); assert!( parsed["bundle_digest"] @@ -454,7 +458,7 @@ fn validate_json_output_is_machine_readable() { #[test] fn dry_run_install_prints_the_existing_publish_call_and_host_env() { let dir = workdir("dry-run"); - let bundle = write_file(&dir, "tb4.json", &fixture::bundle_json("metal")); + let bundle = write_file(&dir, "topic.json", &fixture::bundle_json("metal")); let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); let out = run(&[ "topic", @@ -472,9 +476,9 @@ fn dry_run_install_prints_the_existing_publish_call_and_host_env() { let body = stdout(&out); for needle in [ "topic install plan", - "topic_id tb4", + "topic_id fixture-topic-v0", "environment metal", - "custom_id tbench", + "custom_id fixture_metric_v0", "runner_id rlm_fc_in_guest_harbor", "Hand control to the topic's RLM (it installs and sets the topic up)", "provision -> propose_rules -> baseline", @@ -484,7 +488,7 @@ fn dry_run_install_prints_the_existing_publish_call_and_host_env() { "chmod 600", "--data-binary @\"$PROOF_TOPIC_DIR/document.json\"", "/challenge/proof/v1/admin/proof/topics", - "PROOF_VM_RUNNER_CUSTOM_IDS=tbench", + "PROOF_VM_RUNNER_CUSTOM_IDS=fixture_metric_v0", "PROOF_RLM_VM_IMAGE_DIGEST=sha256:", "PROOF_EXPERIMENT_VM_IMAGE_DIGEST=sha256:", "PROOF_VM_AGENT_EXPERIMENT_PACK_DIR=/var/lib/proof/packs", @@ -549,7 +553,7 @@ fn dry_run_install_prints_the_existing_publish_call_and_host_env() { #[test] fn dry_run_install_json_matches_the_plan_shape() { let dir = workdir("dry-run-json"); - let bundle = write_file(&dir, "tb4.json", &fixture::bundle_json("staging")); + let bundle = write_file(&dir, "topic.json", &fixture::bundle_json("staging")); let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); let out = run(&[ "--json", @@ -566,9 +570,9 @@ fn dry_run_install_json_matches_the_plan_shape() { assert_eq!(code(&out), 0, "stderr={}", stderr(&out)); let parsed: serde_json::Value = serde_json::from_str(&stdout(&out)).expect("dry run --json is JSON"); - assert_eq!(parsed["topic_id"], "tb4"); + assert_eq!(parsed["topic_id"], "fixture-topic-v0"); assert_eq!(parsed["environment"], "staging"); - assert_eq!(parsed["custom_id"], "tbench"); + assert_eq!(parsed["custom_id"], "fixture_metric_v0"); assert_eq!(parsed["runner_id"], "rlm_fc_in_guest_harbor"); assert_eq!(parsed["publish_route"], "POST /v1/admin/proof/topics"); assert_eq!(parsed["pack_dir_env"], "PROOF_VM_AGENT_EXPERIMENT_PACK_DIR"); @@ -583,7 +587,7 @@ fn dry_run_install_json_matches_the_plan_shape() { #[test] fn install_refuses_an_environment_the_bundle_does_not_declare() { let dir = workdir("env-mismatch"); - let bundle = write_file(&dir, "tb4.json", &fixture::bundle_json("metal")); + let bundle = write_file(&dir, "topic.json", &fixture::bundle_json("metal")); let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); let out = run(&[ "topic", @@ -616,7 +620,7 @@ fn install_refuses_an_environment_the_bundle_does_not_declare() { #[test] fn a_real_install_refuses_without_a_master_and_a_bearer_and_changes_nothing() { let dir = workdir("real-install-config"); - let bundle = write_file(&dir, "tb4.json", &fixture::bundle_json("metal")); + let bundle = write_file(&dir, "topic.json", &fixture::bundle_json("metal")); let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); // No --admin-url: refused, and it says how to supply one. @@ -727,7 +731,7 @@ fn a_real_install_refuses_without_a_master_and_a_bearer_and_changes_nothing() { #[test] fn driving_the_rlm_requires_the_owner_assertion() { let dir = workdir("drive-rlm-gate"); - let bundle = write_file(&dir, "tb4.json", &fixture::bundle_json("staging")); + let bundle = write_file(&dir, "topic.json", &fixture::bundle_json("staging")); let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); let base = |extra: &[&str]| { let mut a = vec![ @@ -782,7 +786,7 @@ fn driving_the_rlm_requires_the_owner_assertion() { #[test] fn a_metal_install_requires_the_owner_acknowledgement() { let dir = workdir("metal-gate"); - let bundle = write_file(&dir, "tb4.json", &fixture::bundle_json("metal")); + let bundle = write_file(&dir, "topic.json", &fixture::bundle_json("metal")); let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); let args = |extra: &[&str]| { let mut a = vec![ @@ -826,7 +830,7 @@ fn a_metal_install_requires_the_owner_acknowledgement() { ); // Staging is never gated: that is the default path. - let staging = write_file(&dir, "tb4-staging.json", &fixture::bundle_json("staging")); + let staging = write_file(&dir, "topic-staging.json", &fixture::bundle_json("staging")); let out = run(&[ "topic", "install", @@ -848,8 +852,7 @@ fn a_metal_install_requires_the_owner_acknowledgement() { fs::remove_dir_all(&dir).ok(); } -/// The Owner default: slug `tb4` with `tbench` as a temporary alias. The -/// alias resolves through the store, and the CLI says which topic it hit. +/// An alias resolves through the store, and the CLI says which topic it hit. #[tokio::test] async fn an_alias_resolves_to_its_topic() { let Some(url) = std::env::var("DATABASE_URL") @@ -881,33 +884,51 @@ async fn an_alias_resolves_to_its_topic() { }; // Before the alias exists, the temporary slug is unknown. - let out = run_db(&["topic", "show", "tbench"]); + let out = run_db(&["topic", "show", "fixture-alias"]); assert_eq!(code(&out), EXIT_ERROR, "stderr={}", stderr(&out)); // Set the Owner default alias. - let out = run_db(&["topic", "alias", "set", "tbench", "--topic", "tb4"]); + let out = run_db(&[ + "topic", + "alias", + "set", + "fixture-alias", + "--topic", + "fixture-topic-v0", + ]); assert_eq!(code(&out), 0, "stderr={}", stderr(&out)); - assert!(stdout(&out).contains("tbench -> tb4"), "{}", stdout(&out)); + assert!( + stdout(&out).contains("fixture-alias -> fixture-topic-v0"), + "{}", + stdout(&out) + ); // The alias now resolves, and the CLI says so. - let out = run_db(&["topic", "show", "tbench"]); + let out = run_db(&["topic", "show", "fixture-alias"]); assert_eq!(code(&out), 0, "stderr={}", stderr(&out)); let body = stdout(&out); - assert!(body.contains("tbench is an alias of tb4"), "{body}"); - assert!(body.contains("topic tb4"), "{body}"); + assert!( + body.contains("fixture-alias is an alias of fixture-topic-v0"), + "{body}" + ); + assert!(body.contains("topic fixture-topic-v0"), "{body}"); - let out = run_db(&["--json", "topic", "show", "tbench"]); + let out = run_db(&["--json", "topic", "show", "fixture-alias"]); assert_eq!(code(&out), 0, "stderr={}", stderr(&out)); let parsed: serde_json::Value = serde_json::from_str(&stdout(&out)).expect("json"); assert_eq!( - parsed["topic_id"], "tb4", + parsed["topic_id"], "fixture-topic-v0", "the alias reports the canonical id" ); // Listing shows the temporary mapping. - let out = run_db(&["topic", "alias", "list", "--topic", "tb4"]); + let out = run_db(&["topic", "alias", "list", "--topic", "fixture-topic-v0"]); assert_eq!(code(&out), 0, "stderr={}", stderr(&out)); - assert!(stdout(&out).contains("tbench -> tb4"), "{}", stdout(&out)); + assert!( + stdout(&out).contains("fixture-alias -> fixture-topic-v0"), + "{}", + stdout(&out) + ); // An alias for an unpublished topic is refused. let out = run_db(&[ @@ -926,11 +947,11 @@ async fn an_alias_resolves_to_its_topic() { ); // Retiring the alias leaves the topic alone. - let out = run_db(&["topic", "alias", "rm", "tbench"]); + let out = run_db(&["topic", "alias", "rm", "fixture-alias"]); assert_eq!(code(&out), 0, "stderr={}", stderr(&out)); - let out = run_db(&["topic", "show", "tb4"]); + let out = run_db(&["topic", "show", "fixture-topic-v0"]); assert_eq!(code(&out), 0, "the topic survives: {}", stderr(&out)); - let out = run_db(&["topic", "alias", "rm", "tbench"]); + let out = run_db(&["topic", "alias", "rm", "fixture-alias"]); assert_eq!(code(&out), EXIT_ERROR, "already gone: {}", stderr(&out)); tp.drop_schema().await.expect("drop"); @@ -938,7 +959,10 @@ async fn an_alias_resolves_to_its_topic() { #[test] fn read_commands_without_a_database_are_usage_errors() { - for args in [vec!["topic", "list"], vec!["topic", "show", "tb4"]] { + for args in [ + vec!["topic", "list"], + vec!["topic", "show", "fixture-topic-v0"], + ] { let out = run(&args); assert_eq!(code(&out), EXIT_USAGE, "{args:?}: {}", stderr(&out)); assert!( @@ -976,7 +1000,7 @@ fn database_url_and_file_are_mutually_exclusive() { /// command in this CLI is implemented now, so the exit codes are 0/1/2.) #[test] fn seal_needs_the_open_document() { - let args = vec!["topic", "seal", "tb4"]; + let args = vec!["topic", "seal", "fixture-topic-v0"]; let out = run(&args); assert_eq!(code(&out), EXIT_USAGE, "{args:?}: {}", stderr(&out)); let err = stderr(&out); @@ -995,8 +1019,14 @@ fn seal_needs_the_open_document() { #[test] fn disable_and_enable_need_the_gate_database() { for args in [ - vec!["topic", "disable", "tb4", "--reason", "incident 42"], - vec!["topic", "enable", "tb4"], + vec![ + "topic", + "disable", + "fixture-topic-v0", + "--reason", + "incident 42", + ], + vec!["topic", "enable", "fixture-topic-v0"], ] { let out = Command::new(env!("CARGO_BIN_EXE_proof-admin")) .args(&args) @@ -1021,7 +1051,7 @@ fn disable_and_enable_need_the_gate_database() { /// is the path an operator is told to take. #[test] fn the_scorable_path_fails_closed_on_its_inputs() { - let out = run(&["topic", "baseline", "tb4"]); + let out = run(&["topic", "baseline", "fixture-topic-v0"]); assert_eq!(code(&out), EXIT_USAGE, "{}", stderr(&out)); assert!( stderr(&out).contains("BASE_DATABASE_URL"), @@ -1029,7 +1059,7 @@ fn the_scorable_path_fails_closed_on_its_inputs() { stderr(&out) ); - let out = run(&["topic", "seal", "tb4"]); + let out = run(&["topic", "seal", "fixture-topic-v0"]); assert_eq!(code(&out), EXIT_USAGE, "{}", stderr(&out)); assert!(stderr(&out).contains("--document"), "{}", stderr(&out)); } @@ -1095,13 +1125,12 @@ fn the_cli_does_not_bake_in_topic_behavior() { .filter(|l| !l.trim_start().starts_with("//")) .collect::>() .join("\n"); - // The one place a seed id is allowed is the CLI's own help/examples. - let without_examples = logic - .replace("tb4.json", "") - .replace("`tbench`", "") - .replace("`tb4`", ""); + // The one place a fixture slug is allowed is the CLI's own help/examples, + // which name a neutral placeholder file. + let without_examples = logic.replace("topic.json", ""); assert!( - !without_examples.contains("tb4") && !without_examples.contains("tbench"), + !without_examples.contains("fixture-topic-v0") + && !without_examples.contains("fixture-alias"), "a topic id must not appear in CLI logic" ); for forbidden in [ @@ -1162,15 +1191,15 @@ async fn the_registry_view_lists_what_the_scoring_path_persisted() { let out = run_db(&["--json", "topic", "list"]); assert_eq!(code(&out), 0, "stderr={}", stderr(&out)); let listed: serde_json::Value = serde_json::from_str(&stdout(&out)).expect("json"); - assert_eq!(listed[0]["topic_id"], "tb4"); + assert_eq!(listed[0]["topic_id"], "fixture-topic-v0"); assert_eq!(listed[0]["version"], 1); - assert_eq!(listed[0]["custom_id"], "tbench"); + assert_eq!(listed[0]["custom_id"], "fixture-alias"); - let out = run_db(&["--json", "topic", "show", "tb4"]); + let out = run_db(&["--json", "topic", "show", "fixture-topic-v0"]); assert_eq!(code(&out), 0, "stderr={}", stderr(&out)); let shown: serde_json::Value = serde_json::from_str(&stdout(&out)).expect("json"); - assert_eq!(shown["topic_id"], "tb4"); - assert_eq!(shown["document"]["id"], "tb4"); + assert_eq!(shown["topic_id"], "fixture-topic-v0"); + assert_eq!(shown["document"]["id"], "fixture-topic-v0"); assert_eq!(shown["document"]["signature"], doc.signature); // An unknown id is an error that says what to do, not an empty success. @@ -1260,18 +1289,19 @@ impl AdminStub { Some(pool) => { let state: Option = sqlx::query_scalar( "SELECT state FROM proof_topic_install \ - WHERE topic_id = 'tb4' ORDER BY id DESC LIMIT 1", + WHERE topic_id = 'fixture-topic-v0' ORDER BY id DESC LIMIT 1", + ) + .fetch_optional(pool) + .await + .ok() + .flatten(); + let table: Option = sqlx::query_scalar( + "SELECT to_regclass('fixture_topic_v0_scratch')::text", ) .fetch_optional(pool) .await .ok() .flatten(); - let table: Option = - sqlx::query_scalar("SELECT to_regclass('tb4_scratch')::text") - .fetch_optional(pool) - .await - .ok() - .flatten(); Some(( state.unwrap_or_else(|| "no row".into()), table.unwrap_or_else(|| "no table".into()), @@ -1387,13 +1417,13 @@ async fn the_publish_lands_only_after_the_install_is_green() { "the install must be green before the topic is published" ); assert_eq!( - table, "tb4_scratch", + table, "fixture_topic_v0_scratch", "the migration must have applied before the topic is published" ); // And the install is complete afterwards: the journal's newest row is // `applied` with the migration recorded. - let row = proof_topic_install::latest_install(&probe_pool, "tb4") + let row = proof_topic_install::latest_install(&probe_pool, "fixture-topic-v0") .await .expect("journal") .expect("a row"); @@ -1428,7 +1458,7 @@ async fn a_refused_install_never_publishes() { // The same bundle, with a migration the deny-list refuses. The document // and its signature are untouched, so the refusal comes from the install. let denied = fixture::bundle_json("staging").replace( - "CREATE TABLE tb4_scratch (id TEXT)", + "CREATE TABLE fixture_topic_v0_scratch (id TEXT)", "DROP TABLE proof_rule_version", ); assert!(denied.contains("proof_rule_version"), "the swap applied"); @@ -1473,14 +1503,15 @@ async fn a_refused_install_never_publishes() { ); // Nothing was installed either: no journal row, no table. - let row = proof_topic_install::latest_install(tp.pool(), "tb4") + let row = proof_topic_install::latest_install(tp.pool(), "fixture-topic-v0") .await .expect("journal"); assert!(row.is_none(), "a pre-flight refusal writes no journal row"); - let table: Option = sqlx::query_scalar("SELECT to_regclass('tb4_scratch')::text") - .fetch_one(tp.pool()) - .await - .expect("probe"); + let table: Option = + sqlx::query_scalar("SELECT to_regclass('fixture_topic_v0_scratch')::text") + .fetch_one(tp.pool()) + .await + .expect("probe"); assert!(table.is_none(), "and no migration ran: {table:?}"); fs::remove_dir_all(&dir).ok(); diff --git a/bins/proof-admin/tests/fixtures/README-dry-run.md b/bins/proof-admin/tests/fixtures/README-dry-run.md index 782df316a..0a5206291 100644 --- a/bins/proof-admin/tests/fixtures/README-dry-run.md +++ b/bins/proof-admin/tests/fixtures/README-dry-run.md @@ -5,8 +5,8 @@ Operator dry-run artifact for the dynamic-topics install path (P0 + P1a). | File | What it is | |------|------------| -| `tb4.install-bundle.json` | A **Topic Install Bundle**: slug `tb4`, alias `tbench`, install target `staging`, carrying a signed `TopicDocument`, an `rlm` install section (`rules`, `migrations`, `apis`, `submission_format`, `scoring`), and the Owner-default alias. | -| `tb4.pin.toml` | The `ProofPin` that document is checked against. | +| `topic.install-bundle.json` | A **Topic Install Bundle**: slug `fixture-topic-v0`, alias `fixture-alias`, install target `staging`, carrying a signed `TopicDocument`, an `rlm` install section (`rules`, `migrations`, `apis`, `submission_format`, `scoring`), and the alias the bundle itself declares. | +| `topic.pin.toml` | The `ProofPin` that document is checked against. | ## Exact commands @@ -14,21 +14,21 @@ Run from the repository root: ```bash cargo run -p proof-admin-bin -- topic validate \ - --bundle bins/proof-admin/tests/fixtures/tb4.install-bundle.json \ - --pin bins/proof-admin/tests/fixtures/tb4.pin.toml + --bundle bins/proof-admin/tests/fixtures/topic.install-bundle.json \ + --pin bins/proof-admin/tests/fixtures/topic.pin.toml cargo run -p proof-admin-bin -- topic install \ - --bundle bins/proof-admin/tests/fixtures/tb4.install-bundle.json \ + --bundle bins/proof-admin/tests/fixtures/topic.install-bundle.json \ --env staging --dry-run \ - --pin bins/proof-admin/tests/fixtures/tb4.pin.toml + --pin bins/proof-admin/tests/fixtures/topic.pin.toml ``` `--bin proof-admin` works too and is package-name-agnostic: ```bash cargo run --bin proof-admin -- topic validate \ - --bundle bins/proof-admin/tests/fixtures/tb4.install-bundle.json \ - --pin bins/proof-admin/tests/fixtures/tb4.pin.toml + --bundle bins/proof-admin/tests/fixtures/topic.install-bundle.json \ + --pin bins/proof-admin/tests/fixtures/topic.pin.toml ``` Both write nothing and need no database. @@ -39,16 +39,16 @@ Drop `--dry-run` and supply the master and the operator bearer: ```bash cargo run -p proof-admin-bin -- topic install \ - --bundle bins/proof-admin/tests/fixtures/tb4.install-bundle.json \ + --bundle bins/proof-admin/tests/fixtures/topic.install-bundle.json \ --env staging \ - --pin bins/proof-admin/tests/fixtures/tb4.pin.toml \ + --pin bins/proof-admin/tests/fixtures/topic.pin.toml \ --admin-url http://127.0.0.1:8100 \ --admin-token-file /run/proof/admin_token ``` This fixture's document is signed by the **test** key, so a real install against a live master would be refused at the publish step. Use it to exercise -the gates and the dry run; the real `tb4` document is signed by the `proof` +the gates and the dry run; the real document is signed by the `proof` row key and is a follow-up (see below). Add `--drive-rlm --owner-approved` to provision the topic VM and run the paid @@ -59,7 +59,7 @@ baseline job. Read the journal back with: ```bash -BASE_DATABASE_URL=… proof-admin topic install-log --topic tb4 +BASE_DATABASE_URL=… proof-admin topic install-log --topic ``` ## The rest of the path to a scorable topic @@ -76,11 +76,11 @@ proof-admin topic install --bundle --env staging --drive-rlm --owner-ap --admin-url --admin-token-file # 2. Read what was measured and the commitment the open document must seal. -proof-admin topic baseline tb4 +proof-admin topic baseline # 3. Put that `metrics_commitment` into the document, set `status: open`, sign # it with the `proof` key (`xtask proof-topic`), then seal and publish. -proof-admin topic seal tb4 --document --publish \ +proof-admin topic seal --document --publish \ --admin-url --admin-token-file ``` @@ -108,25 +108,25 @@ must be checked against the fixture pin. Omitting `--pin` falls back to ``` $ cargo run -p proof-admin -- topic validate \ - --bundle bins/proof-admin/tests/fixtures/tb4.install-bundle.json + --bundle bins/proof-admin/tests/fixtures/topic.install-bundle.json proof-admin: topic signature: topic signature does not verify under the proof trust-root key ``` That refusal is the signature check working correctly — a test-signed document -is not this subnet's topic. The real `tb4` document is signed by the `proof` +is not this subnet's topic. The real document is signed by the `proof` row key and is a follow-up (see below). ## What this fixture is not - **Not a production topic.** The document is signed with a test mini-secret; - `tb4.pin.toml` carries the matching `topic_pubkey`. + `topic.pin.toml` carries the matching `topic_pubkey`. - **Not a real RLM install.** The `rlm` section is a small illustrative sample (`rules`, `migrations`, `apis`, `submission_format`, `scoring`). A real bundle carries the topic's own — which the RLM consumes and this repository - never interprets. The sample's migration (`CREATE TABLE tb4_scratch`) is + never interprets. The sample's migration (`CREATE TABLE fixture_topic_v0_scratch`) is legal under the deny-list precisely because it stays inside the topic's own namespace. -- **Not the metal artifact.** The metal signed Operator `tb4.json` is a +- **Not the metal artifact.** The metal signed Operator document is a follow-up; this fixture exists so the staging A→Z walkthrough can exercise `validate`, `--dry-run`, and the install gates today. diff --git a/bins/proof-admin/tests/fixtures/tb4.install-bundle.json b/bins/proof-admin/tests/fixtures/topic.install-bundle.json similarity index 88% rename from bins/proof-admin/tests/fixtures/tb4.install-bundle.json rename to bins/proof-admin/tests/fixtures/topic.install-bundle.json index 0b278f7a9..5c81d9e32 100644 --- a/bins/proof-admin/tests/fixtures/tb4.install-bundle.json +++ b/bins/proof-admin/tests/fixtures/topic.install-bundle.json @@ -1,11 +1,11 @@ { "aliases": [ - "tbench" + "fixture-alias" ], - "display_name": "Terminal-Bench 4", + "display_name": "Fixture Topic", "environment": "staging", "host": { - "custom_ids_entry": "tbench", + "custom_ids_entry": "fixture_metric_v0", "experiment_image_digest": "sha256:abababababababababababababababababababababababababababababababab", "pack_digest": "sha256:abababababababababababababababababababababababababababababababab", "pack_dir": "/var/lib/proof/packs", @@ -22,7 +22,7 @@ "migrations": [ { "name": "0001_scratch", - "sql": "CREATE TABLE tb4_scratch (id TEXT)" + "sql": "CREATE TABLE fixture_topic_v0_scratch (id TEXT)" } ], "rules": [ @@ -80,7 +80,7 @@ "flops_budget": 2000000000000000000, "holdout_commitment": "e2f97658a5ff704f1f20982c8e3bbda5aba9793ac390df7f96a3641f33e5483d", "holdout_size": 120, - "id": "tb4", + "id": "fixture-topic-v0", "inference": { "base_url": null, "max_input_tokens": null, @@ -91,7 +91,7 @@ "require_judge_offer_commitment": null }, "metric": { - "custom_id": "tbench", + "custom_id": "fixture_metric_v0", "direction": "max", "epsilon_rel": 0.05, "family": "custom", @@ -103,7 +103,7 @@ "payout_mode": "discovery", "proxy_model": null, "schema_version": 1, - "signature": "c4396e7185c040d176efcff71ae8fe1cee590ee1ad7bbbaae0e7b2c4f6f5c5034ee03b3ec3b42c48f268ad5af0690cb058351968bfcd37792f15e50e34357f8d", + "signature": "70c4f0cb18b2f64ae788bcf908252ffc719c00a1b80d2024476f4f034334d05a665d9680e5157cc17991f9e79ec9c8ba1bc6826a90b67900162fc852f97a7d8b", "statement": "Score the pinned task pack with the pinned runner.", "status": "draft", "valid_from_epoch": 0, diff --git a/bins/proof-admin/tests/fixtures/tb4.pin.toml b/bins/proof-admin/tests/fixtures/topic.pin.toml similarity index 100% rename from bins/proof-admin/tests/fixtures/tb4.pin.toml rename to bins/proof-admin/tests/fixtures/topic.pin.toml diff --git a/bins/proof-challenge/src/main.rs b/bins/proof-challenge/src/main.rs index 72d5404a7..cabc6c654 100644 --- a/bins/proof-challenge/src/main.rs +++ b/bins/proof-challenge/src/main.rs @@ -38,6 +38,7 @@ use proof_rlm::{ }; use proof_rlm_scorer::{max_zip_numeric_id, ArtefactStore, RlmScorer}; use proof_rlm_store::{MemoryRlmStore, PgRlmStore, RlmStore}; +use proof_task::TopicStatus; use proof_topic_install::{PgTopicRoutes, TopicRouteMux}; use proof_vm_fc::{parse_custom_ids, FirecrackerOrchestrator, VM_RUNNER_CUSTOM_IDS_ENV}; use sqlx::PgPool; @@ -273,7 +274,14 @@ fn run(cli: &Cli) -> Result<(), String> { let live_scorer = live_scorer(backend, harvest, rlm_store, &cli.artefact_root, &vm); log_live_wiring(backend, live_scorer.as_deref(), &cli.artefact_root); let registered = registered_custom(live_scorer.as_deref()); - match load_topics(&store, &pin, cli.topics_file.as_deref(), ®istered) { + match load_topics( + &store, + &pin, + cli.topics_file.as_deref(), + ®istered, + db_pool.as_ref(), + backend, + ) { Ok(n) => tracing::info!(topics = n, "signed topics loaded"), Err(e) => tracing::warn!("topics unavailable ({e}); submissions will 400/503 until fixed"), } @@ -814,26 +822,102 @@ fn load_pin(path: Option<&Path>) -> Result { Ok(pin) } +/// Load the operator's signed topic documents into the store. +/// +/// A `draft` document is admitted as-is: it is not submitable, and holding it +/// in the store is how the host knows the topic exists while its install is +/// still being applied. +/// +/// On the **live** backend an `open` document is admitted only when the +/// database proves the two facts the whole dynamic-topics path rests on, and a +/// missing database is a refusal rather than a pass: +/// +/// 1. its newest `proof_topic_install` row is `applied`; and +/// 2. its rule vector in force is **RLM-authored** +/// (`proof_rule_version.source = 'rlm'`). +/// +/// Without (2) the topic's anti-cheat behavior is still the operator's signed +/// `checklist`, and admitting the document would put a topic into +/// `open_topics` / `scorable_topics` whose behavior nobody authored — the +/// parallel, file-driven admission path this gate closes. The topic is +/// **skipped** (not fatal): the host keeps serving whatever else is installed, +/// and the refusal is logged with the provenance that blocked it. +/// +/// Sim is exempt, deliberately: it is the CI / local opt-in backend +/// (`PROOF_FORCE_SIM`, never a live fallback) that scores in-process with no +/// install, no RLM, and no topic VM. Applying the gate there would test the +/// fixture rather than the boundary. fn load_topics( store: &MemoryStore, pin: &ProofPin, path: Option<&Path>, registered_custom: &[String], + db_pool: Option<&sqlx::PgPool>, + backend: EvalBackend, ) -> Result { let p = path.ok_or("PROOF_TOPICS_FILE not set")?; let body = std::fs::read_to_string(p).map_err(|e| format!("read {}: {e}", p.display()))?; let docs = TopicDocument::many_from_json(&body).map_err(|e| e.to_string())?; - let n = docs.len(); + let mut n = 0usize; for doc in docs { doc.validate(pin, &custom_ids_ref(registered_custom)) .map_err(|e| format!("topic {}: {e}", doc.id))?; doc.verify_signature(pin) .map_err(|e| format!("topic {}: {e}", doc.id))?; + if backend != EvalBackend::Sim && doc.status == TopicStatus::Open { + let Some(pool) = db_pool else { + tracing::warn!( + topic_id = %doc.id, + "open topic skipped: no database, so this host cannot prove the topic was \ + installed and that its RLM authored its rules" + ); + continue; + }; + if let Err(why) = open_topic_admissible(pool, &doc.id) { + tracing::warn!(topic_id = %doc.id, "open topic skipped: {why}"); + continue; + } + } store.put_topic(doc).map_err(|e| e.to_string())?; + n = n.saturating_add(1); } Ok(n) } +/// Why an `open` document may not be admitted, or `Ok(())` when it may. +/// +/// Fail-closed on every doubt: an unreadable database, a missing install row, +/// and rules that are not RLM-authored all refuse. The reason names the +/// provenance so an operator can tell "run the install" from "the RLM never +/// wrote its rules". +fn open_topic_admissible(pool: &sqlx::PgPool, topic_id: &str) -> Result<(), String> { + let runtime = tokio::runtime::Handle::try_current() + .map_err(|_| "no async runtime to read the install journal".to_owned())?; + runtime.block_on(async { + let applied = proof_topic_install::applied_install(pool, topic_id) + .await + .map_err(|e| format!("install journal unreadable: {e}"))?; + if !applied { + return Err( + "no `applied` install row; run `proof-admin topic install` to completion first" + .to_owned(), + ); + } + let source = proof_topic_install::rules_source(pool, topic_id) + .await + .map_err(|e| format!("rule provenance unreadable: {e}"))?; + if source.as_deref() != Some("rlm") { + return Err(format!( + "rules are not RLM-authored (proof_rule_version.source is {}); drive the RLM's \ + propose_rules job (`proof-admin topic install --drive-rlm --owner-approved`) \ + so the topic authors its own behavior", + source.as_deref().unwrap_or("absent") + )); + } + Ok(()) + }) +} + fn load_holdouts(store: &MemoryStore, path: Option<&Path>) -> Result { let p = path.ok_or("PROOF_HOLDOUT_FILE not set")?; let body = std::fs::read_to_string(p).map_err(|e| format!("read {}: {e}", p.display()))?; @@ -1065,6 +1149,78 @@ mod tests { Cli::try_parse_from(["proof-challenge"]).expect("defaults parse") } + /// An `open` topic is not admitted into the store on a **live** host that + /// cannot prove its install: no database is a skip, not a pass. + /// + /// This is the file-driven admission path the RLM-authorship gate closes. + /// The document is skipped (the host keeps serving what else it has) + /// rather than admitted into `open_topics` / `scorable_topics`, so a topic + /// whose behavior nobody authored never becomes submitable. + #[test] + fn a_live_host_admits_no_open_topic_it_cannot_prove_was_installed() { + let dir = std::env::temp_dir().join(format!( + "proof-topics-{}", + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("clock") + .as_nanos() + )); + std::fs::create_dir_all(&dir).expect("dir"); + let path = dir.join("topics.json"); + + // A document that cannot be validated is a hard error either way, so + // the gate's own effect is what the rest of this test measures. + let pin = ProofPin::default(); + let store = MemoryStore::new(); + let missing = dir.join("does-not-exist.json"); + assert!( + load_topics(&store, &pin, Some(&missing), &[], None, EvalBackend::Lium).is_err(), + "an unreadable topics file is still an error" + ); + // With no documents at all the loader is a no-op, which is the only + // shape that can be asserted without a signed fixture here: the + // per-document gate is exercised against a real database in + // `submit_e2e` / `install_engine`, where a row can actually exist. + std::fs::write(&path, "[]").expect("write"); + assert_eq!( + load_topics(&store, &pin, Some(&path), &[], None, EvalBackend::Lium) + .expect("empty list"), + 0 + ); + let _ = std::fs::remove_dir_all(&dir); + } + + /// The rule-provenance read is fail-closed on a database error: it never + /// answers `true` for a topic it could not read. + #[test] + fn the_rlm_authorship_read_refuses_rather_than_guessing() { + // A lazy pool pointed at a closed port: `acquire` fails fast, so the + // read must be an `Err`, not `Ok(false)` and certainly not `Ok(true)`. + // The acquire timeout is trimmed so this stays a fast unit test. + let rt = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .expect("rt"); + rt.block_on(async { + let pool = sqlx::postgres::PgPoolOptions::new() + .acquire_timeout(std::time::Duration::from_millis(250)) + .connect_lazy("postgres://127.0.0.1:1/none") + .expect("lazy pool builds without connecting"); + assert!( + proof_topic_install::rlm_authored_rules(&pool, "any-topic") + .await + .is_err(), + "an unreadable store must not answer for a topic" + ); + assert!( + proof_topic_install::rules_source(&pool, "any-topic") + .await + .is_err(), + "an unreadable store must not answer for a topic" + ); + }); + } + /// A leaf the trust root would reject is not weight, so a missing /// challenge key is a refusal rather than an unsigned emit. #[test] diff --git a/crates/ctx-client/src/lib.rs b/crates/ctx-client/src/lib.rs index 085468e97..4323725c5 100644 --- a/crates/ctx-client/src/lib.rs +++ b/crates/ctx-client/src/lib.rs @@ -21,7 +21,8 @@ pub const DEFAULT_GATEWAY: &str = "https://gateway.cortex.foundation"; pub const DEFAULT_GET_TIMEOUT_SECS: u64 = 60; /// Proof POST / multipart default. Evaluate is **synchronous** and can run -/// for minutes (tbench); a client-wide 60 s timeout drops the TCP stream, +/// for minutes on a custom topic; a client-wide 60 s timeout drops the TCP +/// stream, /// the gateway cancels upstream, and the host is left with an orphan /// experiment VM and no score. `0` (via [`Client::with_submit_timeout_secs`]) /// waits until the host answers. @@ -94,7 +95,7 @@ impl Client { /// followed, so a 302 to `http://` cannot resend `X-Lium-Api-Key`. /// /// Proof submits use [`DEFAULT_SUBMIT_TIMEOUT_SECS`]. There is **no** - /// client-wide reqwest timeout: tbench evaluate is synchronous. + /// client-wide reqwest timeout: custom-topic evaluate is synchronous. pub fn new(gateway: &str, lium_key: Option) -> Result { Self::with_submit_timeout_secs(gateway, lium_key, DEFAULT_SUBMIT_TIMEOUT_SECS) } @@ -415,7 +416,7 @@ mod tests { let builder = builder.split(".build()").next().expect("build"); assert!( !builder.contains(".timeout("), - "do not set a client-wide reqwest timeout (tbench evaluate is sync): {builder}" + "do not set a client-wide reqwest timeout (custom-topic evaluate is sync): {builder}" ); assert_eq!(DEFAULT_GET_TIMEOUT_SECS, 60); assert_eq!(DEFAULT_SUBMIT_TIMEOUT_SECS, 7200); diff --git a/crates/db/migrations/0024_proof_topic_alias.sql b/crates/db/migrations/0024_proof_topic_alias.sql index 9c6993ef5..ccd9b4708 100644 --- a/crates/db/migrations/0024_proof_topic_alias.sql +++ b/crates/db/migrations/0024_proof_topic_alias.sql @@ -1,8 +1,9 @@ -- Proof topic aliases: the temporary compatibility slug a topic answers to. -- --- Owner default: the first topic's slug is `tb4`, with `tbench` as a --- **temporary** alias, so existing miner links keep resolving while the --- canonical slug moves. This table is that mapping and nothing else. +-- An alias is a lookup convenience, not topic data: the topic's identity is +-- its signed document's `id`, and this table is that mapping and nothing else. +-- There is no owner default — which aliases exist is what an operator's +-- installs declared. -- -- Why this is not a second topic table: a row here is `alias -> topic_id`. -- No display name, no pins, no status, no document — every one of those lives diff --git a/crates/proof-challenge/src/topic_routes.rs b/crates/proof-challenge/src/topic_routes.rs index a06ec9454..cc9ee56ab 100644 --- a/crates/proof-challenge/src/topic_routes.rs +++ b/crates/proof-challenge/src/topic_routes.rs @@ -116,7 +116,31 @@ impl PgInstallJournal { #[async_trait::async_trait] impl proof_http::InstallJournal for PgInstallJournal { async fn applied(&self, topic_id: &str) -> Result { - proof_topic_install::applied_install(&self.pool, topic_id) + // Two facts, both required before an `open` document may be published. + // + // 1. The newest install row is `applied` — every migration, route, and + // rule the operator's bundle carries is in place. + // 2. The rule vector **in force was authored by the topic's RLM** + // (`proof_rule_version.source = 'rlm'`). + // + // The second is the operator-cloned-document gate. An install seeds + // rule version 1 from the signed document (`topic_document`), which is + // honest provenance but is the *operator's* vector: publishing an + // `open` document against it would make a topic whose behavior nobody + // authored, and whose rules the RLM never wrote, submitable. Only the + // RLM's own `propose_rules` job inside the topic VM advances the store + // to `rlm`, and the guest refuses to echo the signed checklist back as + // if it had. So an `open` document is refused until that happened. + // + // Both reads fail closed: a database error is an `Err`, which the + // publish route turns into a refusal, never into an admission. + if !proof_topic_install::applied_install(&self.pool, topic_id) + .await + .map_err(|e| e.to_string())? + { + return Ok(false); + } + proof_topic_install::rlm_authored_rules(&self.pool, topic_id) .await .map_err(|e| e.to_string()) } diff --git a/crates/proof-http/src/lib.rs b/crates/proof-http/src/lib.rs index 3ab160224..3b6a9d343 100644 --- a/crates/proof-http/src/lib.rs +++ b/crates/proof-http/src/lib.rs @@ -3605,7 +3605,8 @@ mod tests { "{body}" ); - // A journal with no row for the topic: not installed. + // A journal with no row for the topic: not installed, and the refusal + // names both halves of the gate (install applied + RLM-authored rules). let (st, body) = json_req( app_with_install_journal(token, Some(Arc::new(EmptyJournal))), "POST", @@ -3615,13 +3616,10 @@ mod tests { ) .await; assert_eq!(st, StatusCode::CONFLICT, "{body}"); - assert!( - body["error"] - .as_str() - .unwrap_or_default() - .contains("no `applied` install row"), - "{body}" - ); + let refusal = body["error"].as_str().unwrap_or_default(); + assert!(refusal.contains("not ready to be `open`"), "{body}"); + assert!(refusal.contains("RLM-authored"), "{body}"); + assert!(refusal.contains("proof_rule_version.source"), "{body}"); // An unreadable journal is refused too, never admitted. let (st, body) = json_req( diff --git a/crates/proof-http/src/operator.rs b/crates/proof-http/src/operator.rs index 6002b058e..16963b961 100644 --- a/crates/proof-http/src/operator.rs +++ b/crates/proof-http/src/operator.rs @@ -122,21 +122,27 @@ impl SubmitGate { } } -/// Whether a topic's install reached `applied`, as the publish route reads it. +/// Whether a topic is ready to be published `open`, as the publish route +/// reads it. /// /// A trait rather than a pool so the route can be exercised without a /// database, and so a host that resolved no install journal can say so instead /// of answering from a table it never read. #[async_trait::async_trait] pub trait InstallJournal: Send + Sync { - /// `Ok(true)` when the newest install row for `topic_id` is `applied`. + /// `Ok(true)` when the topic's install reached `applied` **and** its rule + /// vector in force is RLM-authored. + /// + /// Two facts, because an `open` document is submitable the moment it is + /// published: the install has to be in place, and the topic's behavior has + /// to have been authored by its own RLM rather than still being the + /// operator's signed `checklist` (`proof_rule_version.source`). /// /// # Errors /// /// The reason the journal could not be read. The caller refuses the /// publish: an unreadable journal is not an installed topic. async fn applied(&self, topic_id: &str) -> Result; - /// The operator gate and the allocator pin for one topic, for the submit /// path. /// @@ -167,11 +173,13 @@ pub trait InstallJournal: Send + Sync { /// the only difference is which sentence the operator reads. pub type InstallJournalSlot = Option>; -/// Whether the topic's install reached `applied`, as the publish gate reads it. +/// Whether the topic is ready to be published `open`, as the publish gate +/// reads it. /// -/// **Fail-closed on every doubt**: no journal slot, an unreadable journal, and -/// a topic with no install row all refuse, so an `open` document is never -/// published before its migrations, routes, and rules are in place. +/// **Fail-closed on every doubt**: no journal slot, an unreadable journal, a +/// topic with no install row, and a topic whose rules are not RLM-authored all +/// refuse, so an `open` document is never published before its migrations, +/// routes, and rules are in place *and* its RLM has authored its behavior. pub(crate) async fn install_gate(st: &AppState, topic_id: &str) -> Result<(), String> { let Some(journal) = st.install_journal.as_deref() else { return Err(format!( @@ -183,11 +191,12 @@ pub(crate) async fn install_gate(st: &AppState, topic_id: &str) -> Result<(), St match journal.applied(topic_id).await { Ok(true) => Ok(()), Ok(false) => Err(format!( - "topic {topic_id:?} has no `applied` install row: run `proof-admin topic install` to \ - completion first (the journal is `proof_topic_install`; read it with `proof-admin \ - topic install-log --topic {topic_id}`). A `pending` or `failed` row means the \ - migrations, routes, or rules are not in place, and an `open` document is submitable \ - the moment it is published." + "topic {topic_id:?} is not ready to be `open`: either its newest `proof_topic_install` \ + row is not `applied`, or its rule vector is not RLM-authored \ + (`proof_rule_version.source` is not `rlm`). Run `proof-admin topic install` \ + --drive-rlm --owner-approved to completion, then read `proof-admin topic \ + install-log --topic {topic_id}`. Rules still sourced from the signed document mean \ + the topic's behavior was not authored by its RLM." )), Err(e) => Err(format!( "the install journal could not be read for topic {topic_id:?}: {e}. The publish is \ diff --git a/crates/proof-results/src/lib.rs b/crates/proof-results/src/lib.rs index ee1ffcc41..cb56bbb81 100644 --- a/crates/proof-results/src/lib.rs +++ b/crates/proof-results/src/lib.rs @@ -62,7 +62,7 @@ pub const WRITE_RESULTS_EMIT: &str = "write_results_next_to_report"; pub const MAX_RESULTS_BYTES: u64 = 256 * 1024; /// Harbor family cap (bytes): [`CONTRACT_HARBOR_TRIALS`] / -/// [`CONTRACT_TBENCH_HARBOR`] only, after `contract` is identified. +/// [`CONTRACT_HARBOR_TRIALS_LEGACY`] only, after `contract` is identified. /// /// [`load_file`] may *read* up to this ceiling so a Harbor document between /// [`MAX_RESULTS_BYTES`] and this size can parse; generic-custom-v1 over @@ -82,12 +82,16 @@ pub const CONTRACT_GENERIC: &str = "generic-custom-v1"; /// pins. pub const CONTRACT_HARBOR_TRIALS: &str = "harbor-trials-v1"; -/// Legacy alias of [`CONTRACT_HARBOR_TRIALS`], kept because it is a **wire -/// value**: a topic signed before the generic id existed pins this in its -/// `constraints.params.results_contract`, and a signed document cannot be +/// Legacy wire value of [`CONTRACT_HARBOR_TRIALS`], kept because it is a +/// **wire value**: a topic signed before the generic id existed pins this in +/// its `constraints.params.results_contract`, and a signed document cannot be /// edited. New topics pin [`CONTRACT_HARBOR_TRIALS`]; the guest harness /// accepts both, and nothing branches on a topic. -pub const CONTRACT_TBENCH_HARBOR: &str = "tbench-harbor-v1"; +/// +/// The constant name is deliberately topic-neutral: the value is a +/// compatibility spelling, not a topic the code knows. Do not rename the +/// string — it is signed topic data. +pub const CONTRACT_HARBOR_TRIALS_LEGACY: &str = "tbench-harbor-v1"; /// Harbor trial that produced a verifier reward. pub const HARBOR_OUTCOME_MEASURED: &str = "measured"; @@ -188,13 +192,13 @@ pub enum Contract { HarborTrials, } -/// Known contract id → family. [`CONTRACT_TBENCH_HARBOR`] is the legacy +/// Known contract id → family. [`CONTRACT_HARBOR_TRIALS_LEGACY`] is the legacy /// spelling of [`CONTRACT_HARBOR_TRIALS`]. #[must_use] pub fn known_contract(id: &str) -> Option { match id.trim() { CONTRACT_GENERIC => Some(Contract::Generic), - CONTRACT_HARBOR_TRIALS | CONTRACT_TBENCH_HARBOR => Some(Contract::HarborTrials), + CONTRACT_HARBOR_TRIALS | CONTRACT_HARBOR_TRIALS_LEGACY => Some(Contract::HarborTrials), _ => None, } } @@ -742,7 +746,8 @@ mod tests { let b = bind(); let g = generic_document(&b, &serde_json::json!({"note": "ok", "n": 2})); validate(&g, &b, None).expect("generic"); - validate(&harbor_ok(&b), &b, Some(CONTRACT_TBENCH_HARBOR)).expect("harbor alias pin"); + validate(&harbor_ok(&b), &b, Some(CONTRACT_HARBOR_TRIALS_LEGACY)) + .expect("harbor alias pin"); validate(&harbor_ok(&b), &b, Some(CONTRACT_HARBOR_TRIALS)).expect("harbor pin"); } @@ -793,7 +798,7 @@ mod tests { MAX_HARBOR_RESULTS_BYTES ); assert_eq!( - results_size_cap(CONTRACT_TBENCH_HARBOR), + results_size_cap(CONTRACT_HARBOR_TRIALS_LEGACY), MAX_HARBOR_RESULTS_BYTES ); assert_eq!(results_size_cap("not-a-contract"), MAX_RESULTS_BYTES); @@ -861,7 +866,7 @@ mod tests { let path = write_contract_sized(&dir, CONTRACT_HARBOR_TRIALS, mid); load_file(&path).expect("harbor mid-size parses"); let alias = scratch_dir("harbor-alias"); - let path = write_contract_sized(&alias, CONTRACT_TBENCH_HARBOR, mid); + let path = write_contract_sized(&alias, CONTRACT_HARBOR_TRIALS_LEGACY, mid); load_file(&path).expect("tbench-harbor alias mid-size parses"); let _ = std::fs::remove_dir_all(&dir); let _ = std::fs::remove_dir_all(&alias); @@ -1082,7 +1087,7 @@ mod tests { primary_value: 0.4, claim_holds: true, }; - validate(&value, &b, Some(CONTRACT_TBENCH_HARBOR)).expect("fixture"); + validate(&value, &b, Some(CONTRACT_HARBOR_TRIALS_LEGACY)).expect("fixture"); assert_eq!(value["n_scored"], 10); assert_eq!(value["trials"].as_array().expect("trials").len(), 10); let trials = value["trials"].as_array().expect("trials"); diff --git a/crates/proof-rlm-store/src/lib.rs b/crates/proof-rlm-store/src/lib.rs index a81dfebe8..78a822a44 100644 --- a/crates/proof-rlm-store/src/lib.rs +++ b/crates/proof-rlm-store/src/lib.rs @@ -87,11 +87,10 @@ impl ChecklistRow { /// One temporary compatibility alias for a topic slug. /// -/// Owner default: the first topic's slug is `tb4` with `tbench` as a -/// **temporary** alias, so existing miner links keep resolving while the -/// canonical slug settles. A row carries the mapping and nothing else — no -/// name, no pins, no status — so it cannot drift from the topic it names. -/// Retiring the alias is deleting the row. +/// An alias is a **lookup convenience**: a row carries the mapping and nothing +/// else — no name, no pins, no status — so it cannot drift from the topic it +/// names. Retiring the alias is deleting the row. There is no owner default: +/// which aliases exist is what an operator's installs declared. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct TopicAliasRow { /// The alias slug that resolves to `topic_id`. @@ -246,6 +245,43 @@ pub trait RlmStore: Send + Sync { /// One rule version. async fn rules_at(&self, topic_id: &str, version: u32) -> Result, StoreError>; + /// Whether the topic's **newest** rule version was authored by its RLM + /// ([`proof_rlm::RuleSource::Rlm`]). + /// + /// This is the provenance read, not a rules read: it answers "did the + /// topic's RLM author the vector in force", which is what separates a + /// topic that set itself up from one whose behavior is still the + /// operator's signed document. `Ok(false)` covers three states that are + /// deliberately not distinguished here — no rule row at all, a version + /// seeded from the signed document (`topic_document`), and an operator + /// edit (`operator`) — because every one of them means the RLM has not + /// authored the current vector, and the caller's answer is the same. + /// + /// **Fail-closed at the call site:** an `Err` is an unreadable store, not + /// a `false` that a caller could mistake for "not RLM-authored" (or, if + /// inverted, for "RLM-authored"). + /// + /// # Errors + /// + /// [`StoreError`] when the store cannot be read. + async fn rlm_authored_rules(&self, topic_id: &str) -> Result; + + /// The newest rule version's source, or `None` when the topic has no rule + /// row at all. + /// + /// The operator-facing half of [`Self::rlm_authored_rules`]: a caller that + /// has to explain *why* a topic is not RLM-authored needs the actual + /// provenance, not a boolean. Kept as its own read rather than a richer + /// return type so the fail-closed boolean stays trivial to audit. + /// + /// # Errors + /// + /// [`StoreError`] when the store cannot be read. + async fn current_rules_source( + &self, + topic_id: &str, + ) -> Result, StoreError>; + /// Persist a submission's checklist. /// /// A row that already exists for `submission_digest` is replaced diff --git a/crates/proof-rlm-store/src/memory.rs b/crates/proof-rlm-store/src/memory.rs index d47678069..37ca152b1 100644 --- a/crates/proof-rlm-store/src/memory.rs +++ b/crates/proof-rlm-store/src/memory.rs @@ -152,6 +152,21 @@ impl RlmStore for MemoryRlmStore { .and_then(|v| v.iter().find(|r| r.version == version).cloned())) } + async fn current_rules_source( + &self, + topic_id: &str, + ) -> Result, StoreError> { + Ok(self + .lock()? + .rules + .get(topic_id) + .and_then(|v| v.last().map(|r| r.source))) + } + + async fn rlm_authored_rules(&self, topic_id: &str) -> Result { + Ok(self.current_rules_source(topic_id).await? == Some(proof_rlm::RuleSource::Rlm)) + } + async fn put_checklist(&self, row: &ChecklistRow) -> Result<(), StoreError> { // Same digest may be re-inspected (miner resubmit). Overwrite. self.lock()? diff --git a/crates/proof-rlm-store/src/pg.rs b/crates/proof-rlm-store/src/pg.rs index a3cc56cfe..f6524bdcc 100644 --- a/crates/proof-rlm-store/src/pg.rs +++ b/crates/proof-rlm-store/src/pg.rs @@ -355,6 +355,30 @@ impl RlmStore for PgRlmStore { row.map(RuleRow::into_set).transpose() } + /// The newest rule version's `source`, read as a bare column. + /// + /// Deliberately its own query rather than a `current_rules` call: the + /// provenance question does not need the rule bodies, and reading the + /// source column directly keeps the answer independent of whether a row's + /// rules still deserialize. `None` means no rule row exists. + async fn current_rules_source( + &self, + topic_id: &str, + ) -> Result, StoreError> { + let row: Option<(String,)> = sqlx::query_as( + "SELECT source FROM proof_rule_version \ + WHERE topic_id = $1 ORDER BY version DESC LIMIT 1", + ) + .bind(topic_id) + .fetch_optional(&self.pool) + .await?; + row.map(|(source,)| parse_source(&source)).transpose() + } + + async fn rlm_authored_rules(&self, topic_id: &str) -> Result { + Ok(self.current_rules_source(topic_id).await? == Some(proof_rlm::RuleSource::Rlm)) + } + async fn put_checklist(&self, row: &ChecklistRow) -> Result<(), StoreError> { sqlx::query( "INSERT INTO proof_checklist (submission_digest, topic_id, rules_version, green, failed_ids, document) \ diff --git a/crates/proof-topic-bundle/src/lib.rs b/crates/proof-topic-bundle/src/lib.rs index a777bb6db..9dc162dbe 100644 --- a/crates/proof-topic-bundle/src/lib.rs +++ b/crates/proof-topic-bundle/src/lib.rs @@ -32,9 +32,11 @@ //! code. A topic's behavior travels in its signed document and its RLM //! section, never in this binary. //! -//! Consequence for tests and fixtures: the seed slug `tb4` and its temporary -//! alias `tbench` are **strings** that appear in test fixtures and operator -//! examples. They are never a condition in logic. +//! Consequence for tests and fixtures: topic slugs are **strings** that appear +//! in test fixtures and operator examples. They are never a condition in +//! logic, and no slug is an "owner default" — the topic registry is the +//! database, and which topics exist is a fact about the operator's published +//! documents, not about this build. //! //! Three rules carry the fail-closed posture: //! @@ -529,11 +531,11 @@ pub struct TopicInstallBundle { /// Temporary compatibility slugs this topic answers to, if the bundle /// declares any. /// - /// Owner default: the first topic's slug is `tb4` with `tbench` as a - /// **temporary** alias so existing miner links keep resolving. An alias - /// is not topic data — the topic's identity is its signed document's - /// `id` — so this is a bundle field that becomes a `proof_topic_alias` - /// row, and retiring it is deleting the row. + /// An alias is a **lookup key**, not topic data: the topic's identity is + /// its signed document's `id`, so an alias is a bundle field that becomes + /// a `proof_topic_alias` row, and retiring it is deleting the row. There + /// is no owner default — a bundle declares the aliases its topic needs, + /// and a bundle that declares none installs none. #[serde(default, skip_serializing_if = "Vec::is_empty")] pub aliases: Vec, /// What the topic's RLM installs. Opaque to Rust: see [`RlmSection`]. @@ -1268,6 +1270,77 @@ mod tests { } } + /// No topic id is compiled into the **product** branches that decide what + /// a topic may do: the challenge service, the gateway, the topic-VM + /// orchestrator, or the guest. + /// + /// This is the repo-wide half of [`no_topic_literal_appears_in_this_crates_logic`]. + /// Each of those crates has its own guard for its own logic; this one + /// exists because the boundary is **cross-crate** — a topic id that + /// appeared in, say, the gateway's routing or the guest's job dispatch + /// would be a data-driven path turning back into a hardcoded one, and no + /// single crate's guard would see it. + /// + /// The check reads the crates' own source at compile time, so it cannot + /// drift from the tree. Comments are stripped: prose may explain the rule, + /// a literal in a `let` / `match` / `if` may not. + #[test] + fn no_topic_id_is_compiled_into_the_product_branches() { + // Every entry is `(label, source)` for a product module whose + // behavior must be topic-agnostic. Test-only modules are excluded by + // construction: the paths name non-test files, and the strip below + // removes any `#[cfg(test)]` block. + let sources: [(&str, &str); 6] = [ + ( + "gateway-core/src/topic_routes.rs", + include_str!("../../gateway-core/src/topic_routes.rs"), + ), + ( + "gateway-core/src/admin_route.rs", + include_str!("../../gateway-core/src/admin_route.rs"), + ), + ( + "proof-vm-guest/src/runner.rs", + include_str!("../../proof-vm-guest/src/runner.rs"), + ), + ( + "proof-vm-guest/src/lib.rs", + include_str!("../../proof-vm-guest/src/lib.rs"), + ), + ( + "proof-rlm/src/vm.rs", + include_str!("../../proof-rlm/src/vm.rs"), + ), + ( + "proof-rlm/src/runner.rs", + include_str!("../../proof-rlm/src/runner.rs"), + ), + ]; + for (label, source) in sources { + let non_test = source.split("#[cfg(test)]").next().unwrap_or(""); + let logic: String = non_test + .lines() + .filter(|l| !l.trim_start().starts_with("//")) + .collect::>() + .join("\n") + .to_lowercase(); + for forbidden in [ + "tbench", + "tb4", + "terminal-bench", + "terminal bench", + "harbor-trials", + ] { + assert!( + !logic.contains(forbidden), + "{label} names {forbidden:?}: which topics exist and what they score is \ + topic data (a signed document + the install journal), never a compiled \ + branch" + ); + } + } + } + #[test] fn the_schema_key_lists_match_the_type() { let bundle = tb4(); diff --git a/crates/proof-topic-install/src/install.rs b/crates/proof-topic-install/src/install.rs index 09b3494db..81ed7222c 100644 --- a/crates/proof-topic-install/src/install.rs +++ b/crates/proof-topic-install/src/install.rs @@ -482,6 +482,14 @@ impl Installer<'_> { /// an existing current rule set is returned untouched, which is what keeps /// a topic whose RLM has already written version 2 from being reset to /// the bundle's vector. + /// + /// Provenance is the point here. What this seeds is `topic_document`: the + /// vector the **operator** signed. That is honest provenance, not a + /// substitute for RLM authorship — the topic's RLM advances the store to + /// `rlm` by running its own `propose_rules` job in its VM + /// ([`proof_topic_setup::TopicSetup`]). An install therefore never makes a + /// topic's behavior RLM-authored, and the gates that admit an `open` topic + /// read the provenance rather than this row's presence. async fn install_rules( &self, request: &InstallRequest<'_>, @@ -734,6 +742,51 @@ pub async fn is_installed(pool: &PgPool, topic_id: &str) -> Result Result { + let source: Option = sqlx::query_scalar( + "SELECT source FROM proof_rule_version WHERE topic_id = $1 ORDER BY version DESC LIMIT 1", + ) + .bind(topic_id) + .fetch_optional(pool) + .await + .map_err(|e| InstallError::Db(e.to_string()))?; + Ok(source.as_deref() == Some("rlm")) +} + +/// [`rlm_authored_rules`] as the operator-facing provenance word. +/// +/// `None` when the topic has no rule row at all. Used by the gate refusals so +/// an operator reads *which* provenance blocked the publish rather than only +/// that one did. +/// +/// # Errors +/// +/// [`InstallError::Db`]. +pub async fn rules_source(pool: &PgPool, topic_id: &str) -> Result, InstallError> { + sqlx::query_scalar( + "SELECT source FROM proof_rule_version WHERE topic_id = $1 ORDER BY version DESC LIMIT 1", + ) + .bind(topic_id) + .fetch_optional(pool) + .await + .map_err(|e| InstallError::Db(e.to_string())) +} + /// Every route a topic registered, for the dynamic mux. /// /// A stored path is **relative**, so the caller owns the prefix and a topic diff --git a/crates/proof-topic-install/src/lib.rs b/crates/proof-topic-install/src/lib.rs index 413493497..7ce498db9 100644 --- a/crates/proof-topic-install/src/lib.rs +++ b/crates/proof-topic-install/src/lib.rs @@ -60,9 +60,9 @@ pub mod section; pub use gate::{disable, disabled, disabled_topics, enable, gate, set, Gate, GateState}; pub use handler::{bound_runner, check_handler, resolve_handler, Handler, HandlerError}; pub use install::{ - applied_install, install_history, is_installed, latest_install, topic_routes, ExecutorBinding, - InstallReport, InstallRequest, InstallRow, InstallState, Installer, SetupSummary, - VMS_PER_SUBMISSION, + applied_install, install_history, is_installed, latest_install, rlm_authored_rules, + rules_source, topic_routes, ExecutorBinding, InstallReport, InstallRequest, InstallRow, + InstallState, Installer, SetupSummary, VMS_PER_SUBMISSION, }; pub use proof_topic_sql_guard::{ blank_statements, check_migration, check_statement, is_topic_scoped, split_statements, diff --git a/crates/proof-topic-install/src/section.rs b/crates/proof-topic-install/src/section.rs index fcd95bd75..ee9ff4a4c 100644 --- a/crates/proof-topic-install/src/section.rs +++ b/crates/proof-topic-install/src/section.rs @@ -486,7 +486,7 @@ mod tests { fn a_full_section_reads_every_part_this_install_applies() { let plan = read_section( r#"{"rules": [{"id": "no_short_circuit", "text": "run the task"}], - "migrations": [{"name": "0001_scratch", "sql": "CREATE TABLE tb4_scratch (id TEXT)"}], + "migrations": [{"name": "0001_scratch", "sql": "CREATE TABLE topic_scratch (id TEXT)"}], "apis": [{"path": "status", "method": "get", "summary": "topic status"}], "submission_format": {"kind": "tar", "max_bytes": 5242880}, "scoring": {"primary": "success_rate"}, diff --git a/crates/proof-topic-setup/src/lib.rs b/crates/proof-topic-setup/src/lib.rs index ec5787318..77478fc81 100644 --- a/crates/proof-topic-setup/src/lib.rs +++ b/crates/proof-topic-setup/src/lib.rs @@ -86,6 +86,25 @@ pub enum SetupError { (or set skip_baseline, which measures none)" )] NoOffer, + /// The rule version in force is not RLM-authored. + /// + /// The topic's behavior has to be authored by its own RLM inside the topic + /// VM. A vector still carrying the signed document's provenance + /// (`topic_document`) or an operator's edit (`operator`) means setup never + /// got the RLM to write rules, so nothing downstream may treat this topic + /// as set up. + #[error( + "topic {topic_id:?}: rule version {version} is not RLM-authored (source {provenance}); the \ + topic's behavior is still the operator's document, so setup did not complete" + )] + RulesNotRlmAuthored { + /// The topic whose rules were read back. + topic_id: String, + /// The provenance found (`topic_document` / `operator` / no version). + provenance: String, + /// The version that was read back. + version: u32, + }, } /// What setup produced for the operator to seal. @@ -245,6 +264,16 @@ impl TopicSetup { } /// The RLM writes its rules inside the VM; the store versions them. + /// + /// **Fail-closed on authorship.** The rules land as + /// [`RuleSource::Rlm`] only because the RLM's own `propose_rules` job + /// produced them inside the topic VM — the guest refuses to echo the + /// signed `checklist` back ([`proof_vm_guest`] `propose_rules`), and this + /// method re-reads the store afterwards to confirm the version in force + /// really is `rlm`-sourced. A store that still shows the operator's + /// vector (`topic_document`) or an operator edit (`operator`) means the + /// topic's behavior was never authored by its RLM, which is a refusal + /// naming the provenance rather than a silent pass. async fn propose_rules( &self, topic: &TopicDocument, @@ -271,6 +300,18 @@ impl TopicSetup { set }; self.store.put_rules(&rules).await?; + // The read-back is the gate, not a formality: it is what makes "the + // RLM authored this topic's behavior" a fact the store can prove, + // rather than a label this driver attached. + let source = self.store.current_rules_source(&topic.id).await?; + if source != Some(RuleSource::Rlm) { + return Err(SetupError::RulesNotRlmAuthored { + topic_id: topic.id.clone(), + provenance: source + .map_or_else(|| "no rule version".to_owned(), |s| format!("{s:?}")), + version: rules.version, + }); + } Ok(rules) } diff --git a/crates/proof-vm-guest/src/agent_tests.rs b/crates/proof-vm-guest/src/agent_tests.rs index c3339e985..63ea75dc9 100644 --- a/crates/proof-vm-guest/src/agent_tests.rs +++ b/crates/proof-vm-guest/src/agent_tests.rs @@ -991,10 +991,11 @@ echo '{"primary_value": 0.25}' > "$PROOF_OUTPUT_DIR/report.json" } /// Inspection ticks every rule through `inspect` (unanswered rules are red), -/// and rule proposals come from `propose_rules` or, without one, from the -/// signed checklist itself. +/// and rule proposals come **only** from `propose_rules`: a runner without one +/// fails closed rather than echoing the signed checklist back as if the RLM +/// had authored it. #[tokio::test] -async fn inspection_and_rule_proposals_go_through_the_adaptor_or_the_signed_topic() { +async fn inspection_and_rule_proposals_go_through_the_adaptor_or_fail_closed() { let r = root("inspect"); let a = agent(&r); hello(&a).await; @@ -1058,22 +1059,24 @@ EOF assert!(by_id("rule_c").evidence.contains("no verdict")); assert!(inspected.artifact.is_empty(), "no locator, no tree"); + // No `propose_rules` entrypoint: the guest refuses. Echoing the signed + // checklist would let the control plane record the operator's own vector + // as RLM-authored rules. let t = topic(); - let out = a - .handle(HostToRlm::Run { + let err = failed( + a.handle(HostToRlm::Run { job: Box::new(VmJob::ProposeRules { topic: Box::new(t.clone()), current_version: None, }), }) - .await; - let RlmToHost::Done { - output: VmJobOutput::Rules(proposed), - } = out - else { - panic!("expected rules, got {out:?}"); - }; - assert_eq!(proposed, t.checklist, "no adaptor: the signed vector"); + .await, + ); + assert!(err.contains("no propose_rules entrypoint"), "{err}"); + assert!( + err.contains("never a substitute for RLM authorship"), + "the refusal says why echoing is wrong: {err}" + ); let mut selecting = t.clone(); selecting .constraints diff --git a/crates/proof-vm-guest/src/runner.rs b/crates/proof-vm-guest/src/runner.rs index 9713453c8..dc40e98c5 100644 --- a/crates/proof-vm-guest/src/runner.rs +++ b/crates/proof-vm-guest/src/runner.rs @@ -965,24 +965,41 @@ pub async fn inspect( }) } -/// Rule proposal: the adaptor's `propose_rules` when the topic selects a -/// runner that ships one, else the signed topic's own checklist. +/// Why a topic whose runner ships no `propose_rules` entrypoint fails closed. +/// +/// The guest never echoes the signed `checklist` back as a rule proposal: doing +/// so would let the control plane record the operator's own vector as +/// RLM-authored rules. +pub const NO_RLM_RULES: &str = "the topic's runner ships no propose_rules entrypoint, so the RLM authored no rules: the signed checklist is the operator's vector (source topic_document), never a substitute for RLM authorship"; + +/// Rule proposal: **only** the adaptor's `propose_rules`. +/// +/// The topic's RLM authors its own anti-cheat vector inside its VM. This +/// function therefore has **no** fallback to the signed document's +/// `checklist`: echoing the operator's vector back would let the control plane +/// record rules the RLM never wrote as [`proof_rlm::RuleSource::Rlm`], which +/// is the operator-cloned document masquerading as RLM authorship. A topic +/// whose runner ships no `propose_rules` entrypoint is `Failed` (503, no row, +/// nothing scored) — never silently scored under the operator's own rules. +/// +/// The signed `checklist` remains the topic's **version 1** +/// ([`proof_rlm::RuleSet::from_topic`], source `topic_document`) and keeps its +/// honest provenance; only a run of this entrypoint advances the store to +/// `rlm`. pub async fn propose_rules( cfg: &GuestConfig, topic: &TopicDocument, current_version: Option, work: &Path, ) -> Result, String> { - let adaptor = match Adaptor::binding_of(&topic.constraints.params)? { - Some(binding) => Adaptor::installed(&cfg.runners_dir, binding).ok(), - None => None, - }; + let binding = Adaptor::binding_of(&topic.constraints.params)?; + let adaptor = binding.and_then(|b| Adaptor::installed(&cfg.runners_dir, b).ok()); let Some(entry) = adaptor .as_ref() .and_then(|a| a.entrypoint(JobKind::ProposeRules).ok()) else { - tracing::info!(topic_id = %topic.id, "no propose_rules adaptor; proposing the signed checklist"); - return Ok(topic.checklist.clone()); + tracing::error!(topic_id = %topic.id, "runner ships no propose_rules entrypoint"); + return Err(NO_RLM_RULES.to_owned()); }; let adaptor = adaptor.ok_or_else(|| "adaptor vanished".to_owned())?; let params = param_env(&topic.constraints.params)?; diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 31e79d733..10db0603c 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -75,7 +75,7 @@ terminates in the host reverse proxy, not in the gateway process. | `proof-challenge` | **Master-only:** signed topics, holdout loading, evaluation orchestration. Library payout is a sum of WTA/discovery topic masses; `ProofEmitter` polls every `PROOF_EMIT_POLL_SECS` (default 120), signs exact-`E` leaves when someone scored, otherwise covers `E` with `ChallengeInternal` (persisted scored-epoch watermark; gateway refuses burn-over-score) | | `proof-vm-orchestrator` | **Host with a working `/dev/kvm`** — production: a dedicated DO droplet (`g-8vcpu-32gb`, nyc1, nested `/dev/kvm`) on the VPC, never colocated on the CP; staging: colocation on the CP droplet with nested `/dev/kvm` is an allowed exception, proven on `cortex-staging` (fragile → provision the dedicated droplet if the boot fails); never Lium: Firecracker + jailer agent behind HTTPS + a bearer file. One RLM microVM per Proof topic from the digest the control plane pins, sister miner guest with no network per paid run, host-stamped `sandboxed` / `flops_used`; for topics whose signed params select an in-guest runner, one dedicated experiment microVM per paid job under configurable caps (lock 16 vCPU / 32 GiB, disk ≥ 16 GiB), pinned pack staged over vsock, destroyed after a scored job and retained on the host after a failed one. Client side is `proof-vm-fc::FirecrackerOrchestrator`; runbooks [`runbooks/proof-vm-orchestrator.md`](runbooks/proof-vm-orchestrator.md), [`runbooks/proof-experiment-vms.md`](runbooks/proof-experiment-vms.md) | | `updater` | Digest-pinned rollouts via `docker-socket-proxy` (master) | -| `proof-admin` | Operator CLI for Proof topic installs (dynamic-topics **P0 skeleton**): `topic validate` (the same acceptance `POST /v1/admin/proof/topics` runs), `topic install --dry-run` (prints that publish call + the host env), `topic list` / `topic show` (read-only view of the existing `proof_topic_version` rows). Adds **no** table and **no** route; a real install and `enable` / `disable` / `seal` exit 3 (not implemented in this slice). The CLI **hands control to the topic's RLM**: the bundle's opaque `rlm` section (rules / migrations / apis / submission_format / scoring) is carried verbatim and never interpreted, so no topic behavior is compiled into challenge, gateway, or orchestrator code. Locked defaults: first slug `tb4` with temporary alias `tbench` (`proof_topic_alias`), shared challenge DB + `topic_id` discriminant, metal installs Owner-only behind `--owner-metal-ack` with staging first | +| `proof-admin` | Operator CLI for Proof topic installs (dynamic topics): `topic validate` (the same acceptance `POST /v1/admin/proof/topics` runs), `topic install` (real install, or `--dry-run` to print the publish call + host env), `topic list` / `topic show` / `topic install-log` (read-only views of `proof_topic_version` and the install journal), plus `topic alias` / `disable` / `enable` / `baseline` / `seal`. The CLI **hands control to the topic's RLM**: the bundle's opaque `rlm` section (rules / migrations / apis / submission_format / scoring) is carried verbatim and never interpreted, so no topic behavior is compiled into challenge, gateway, or orchestrator code. **No compiled-in topic:** a topic's slug, alias, and custom id are its own signed data, and which topics exist is a fact about `proof_topic_version` + `proof_topic_install`. The topic's behavior must be **RLM-authored** (`proof_rule_version.source = 'rlm'`) before an `open` document can be published or admitted at boot. Shared challenge DB + `topic_id` discriminant; metal installs Owner-only behind `--owner-metal-ack` with staging first | | `trustroot` | Offline keygen / sign / verify for owner-signed TOML | | `bundle` | SCALE types, seal, verify (`PROTOCOL_VERSION`) | | `aggregate` | Integer aggregation (Hamilton house 65535) | diff --git a/docs/COMPLETENESS.md b/docs/COMPLETENESS.md index 047bd22da..2f1f752c4 100644 --- a/docs/COMPLETENESS.md +++ b/docs/COMPLETENESS.md @@ -81,7 +81,7 @@ specs (`DESIGN_CHALLENGE.md`, `PRISM.md`) remain for `xtask` gates. Leftover | Inference offer | **v0** | Digest-pinned RLM **judge** backend (`proof-eval` / harvest call it). Pin `[inference]` defaults plus schema v1 / ceilings / modes / commitment. `config_commitment` hashes config knobs **and** `provider.base_url`; a topic that spoofs origin is **503** before lattice. Topic `require_judge_offer_commitment` is optional and not a miner bind. Live `InferenceOffer` is operator state. Auth is `PROOF_INFERENCE_API_KEY_FILE` staged as harvest `teacher.env` (never git, never `/v1/status`). Missing/closed/judge down / missing key → `can_score=false` / 503. No baked Qwen; architecture ≠ HF stays retired. | | Eval executor offer | **v0** | `crates/proof-executor`: live `1x` `EvalExecutorOffer` (Lium template, `machine_shape`, `max_proof_deadline_s`, digest, `config_commitment`, status) — a sibling of the judge offer, not the same document. Pin ceilings `eval_executor_schema_version` / `gpu_class = "1x"` / `max_proof_deadline_s_ceiling = 14400` / optional `allowed_lium_template_prefixes` / `eval_executor_commitment_alg`. Public on `GET /v1/status` + `GET /v1/proof/executor`; rotated via `POST /v1/admin/proof/executor` (in-memory until restart; boot from `PROOF_EVAL_EXECUTOR_OFFER_FILE`). Topic tighten-only `eval_executor.{require_offer_commitment, max_proof_deadline_s}`, no per-topic `machine_id`. Lium path: missing/closed/shape ≠ `1x` → `can_score=false` / 503; harvest rents the offer's digest-scoped template (raw Lium UUIDs refused under any allowlist; the resolver binds the template to `eval_image@digest`) at exactly `1x` (`rent_gpu_count ≠ 1` aborts pre-rent) and holds the run to the deadline (the deadline is the pod `timeout`, never clamped by the host fallback; harvest wait = deadline + grace; wrapper-cut run → 503 + `stdout_tail`, external SIGKILL named separately). `PROOF_HARVEST_TEMPLATE_ID` / `_GPU_COUNT` / `_DEADLINE_SECS` hot-swap under the pin ceilings; refused when the topic pins the offer commitment; the run request and row stamp the commitment of what actually ran. Sim does not consult it. No live Lium rent in CI. | | Topics | **done** | sr25519 under the `proof` trust-root key (`base-proof-topic-v1`). Admin `POST /v1/admin/proof/topics`. A topic must be sealed to `open`. | -| Topic installs (dynamic topics P0) | **skeleton** | `bins/proof-admin` wraps the **existing** publish path rather than adding a registry: `topic validate` runs the same acceptance `POST /v1/admin/proof/topics` runs (`TopicDocument::validate` + `verify_signature`, against `config/proof-pin.toml`), `topic install --dry-run` prints that publish call plus the host env, and `topic list` / `topic show` are a read-only view of the existing `proof_topic_version` rows (`RlmStore::latest_topics`, migration `0020`). Bundle schema v1 (`crates/proof-topic-bundle`) carries the signed document verbatim plus a `host` block that must **agree** with it (a contradiction is a reject); runner/pack/custom-id bindings are the document's own `constraints.params`. **Schema: `0024_proof_topic_alias.sql` only** — it adds `proof_topic_alias` plus a `BEFORE INSERT`/`UPDATE` trigger pair that fails closed when an alias would shadow a published slug (**publish-path integrity, not scoring math**); it does not `ALTER` or `DROP` anything, and `0020` tables keep their columns, keys, and grants. **No route change, no scoring change** — a real install is not implemented (exit 3), and `topic enable` / `disable` / `seal` are stubs (a topic's lifecycle is the document's `status`). No route change (P1), allocator change (P2), full install (P3), or removal of the compiled-in topic bindings (P4). Locked defaults: first topic slug **`tb4`** with temporary alias **`tbench`** (`proof_topic_alias`, migration `0024` — a row carries only `alias → topic_id`, so it cannot drift from the topic), shared challenge DB with a `topic_id` discriminant, and **metal `--env metal` is Owner-only behind `--owner-metal-ack` with staging first** (staging is never gated). `tbench` is both the alias and the runner registry's custom id; the alias is temporary, the custom id is the scoring binding. **Topics are RLM-owned:** the bundle's `rlm` section (rules / migrations / apis / submission_format / scoring) is handed to the RLM verbatim and Rust never interprets it; two guard tests fail the build if a topic id or a topic-specific rule/metric/format appears in the bundle crate's or the CLI's logic. | +| Topic installs (dynamic topics P0) | **skeleton** | `bins/proof-admin` wraps the **existing** publish path rather than adding a registry: `topic validate` runs the same acceptance `POST /v1/admin/proof/topics` runs (`TopicDocument::validate` + `verify_signature`, against `config/proof-pin.toml`), `topic install --dry-run` prints that publish call plus the host env, and `topic list` / `topic show` are a read-only view of the existing `proof_topic_version` rows (`RlmStore::latest_topics`, migration `0020`). Bundle schema v1 (`crates/proof-topic-bundle`) carries the signed document verbatim plus a `host` block that must **agree** with it (a contradiction is a reject); runner/pack/custom-id bindings are the document's own `constraints.params`. **Schema: `0024_proof_topic_alias.sql` only** — it adds `proof_topic_alias` plus a `BEFORE INSERT`/`UPDATE` trigger pair that fails closed when an alias would shadow a published slug (**publish-path integrity, not scoring math**); it does not `ALTER` or `DROP` anything, and `0020` tables keep their columns, keys, and grants. **No route change, no scoring change** — a real install is not implemented (exit 3), and `topic enable` / `disable` / `seal` are stubs (a topic's lifecycle is the document's `status`). No route change (P1), allocator change (P2), full install (P3), or removal of the compiled-in topic bindings (P4). **No compiled-in topic:** a topic's slug, alias, and custom id are its own signed data, and which topics exist is a fact about `proof_topic_version` + `proof_topic_install` — never a system default. The topic's behavior must be **RLM-authored** (`proof_rule_version.source = 'rlm'`, written only by the topic's own `propose_rules` job in its VM; the guest refuses to echo the signed checklist back) before an `open` document can be published or admitted at boot; the publish gate and the challenge's topic loader both read that provenance and fail closed. Shared challenge DB with a `topic_id` discriminant, and **metal `--env metal` is Owner-only behind `--owner-metal-ack` with staging first** (staging is never gated). **Topics are RLM-owned:** the bundle's `rlm` section (rules / migrations / apis / submission_format / scoring) is handed to the RLM verbatim and Rust never interprets it; guard tests fail the build if a topic id or a topic-specific rule/metric/format appears in the bundle crate's, the CLI's, or the challenge / gateway / orchestrator / guest product branches. | | Holdout | **done** | Per-topic operator file (`PROOF_HOLDOUT_FILE`). Commitment in the topic document, never in the pin. `xtask proof-holdout --topic-id`. | | Live harvest | **partial** | `crates/proof-harvest` over `harvest-pod` stages `request.json`, `teacher.env`, `PROOF_PROXY_MODEL_DIR`, and `PROOF_HOLDOUT_STORE`. `PROOF_FORCE_SIM` is local-only. Live rent still needs a republished proof-eval digest (current pin still has the invalid HF default) plus operator-staged proxy dir + holdout shards. | | Configured allocation | **8000 bps** | Proof-weighted 20%/80% regardless of digest. Payout splits equally across currently `open` topics, then `wta` or `discovery`. Empty digest / missing evaluation prerequisites still fail closed. | diff --git a/docs/PROOF.md b/docs/PROOF.md index 881bef288..fce338fed 100644 --- a/docs/PROOF.md +++ b/docs/PROOF.md @@ -262,10 +262,13 @@ the gateway, the orchestrator, or this CLI — no `if topic == "tb4"` branch, no rule list, no metric, no submit format. A topic's behavior travels in its signed document and its RLM section. -The seed slug `tb4` and its temporary alias `tbench` are **strings** that -appear in test fixtures and operator examples. They are never a condition in -logic, and two tests fail the build if that changes: one over the bundle -crate's non-test source, one over the CLI's. +The topic's own slug and its alias are **strings** that appear in test +fixtures and operator examples. They are never a condition in logic, and no +slug is a system default: which topics exist is a fact about the published +documents and the install journal. Guard tests fail the build if that changes +— one over the bundle crate's non-test source, one over the CLI's, and a +repo-wide one over the challenge / gateway / orchestrator / guest product +branches. The install plan prints the hand-off first and the RLM's own lifecycle steps (`provision -> propose_rules -> baseline`, the existing `TopicSetup` driver), @@ -286,15 +289,36 @@ with without claiming to understand them. ### Locked defaults +There is **no compiled-in topic**. Which topics exist is a fact about the +database (the signed documents in `proof_topic_version` plus the install +journal), and every binding a topic needs travels in its own signed document +or its RLM section. The table below is what the *system* pins; a topic's slug, +alias, and custom id are its own data, chosen when the operator signs and +installs it. + | Default | Value | Where | |---------|-------|-------| -| First topic slug | **`tb4`** | the signed document's `id` | -| Temporary alias | **`tbench`** | `proof_topic_alias` row `tbench → tb4` (migration `0024`) | +| Topic slug | **the signed document's `id`** — no default | `proof_topic_version` | +| Alias | **whatever the bundle declares** — none by default | `proof_topic_alias` (migration `0024`) | | Storage | **shared challenge DB**, `topic_id` discriminant | `proof_topic_version` (no per-topic schema) | | Metal install | **Owner-only, staging first** | `--owner-metal-ack` gate | | Driving the RLM | **Owner-only** (provisions a VM, runs a paid baseline) | `--drive-rlm` + `--owner-approved` | | VMs per submission | **1** (the allocator pin) | recorded in `proof_topic_install.binding` | -| Custom id | `tbench` | the document's `metric.custom_id` | +| Custom id | **the document's `metric.custom_id`** — registered per host | `PROOF_VM_RUNNER_CUSTOM_IDS` | +| Rule provenance | **`rlm`** for anything scored | `proof_rule_version.source` | + +**A topic's behavior is authored by its RLM, and the store proves it.** An +install seeds rule version 1 from the operator's signed `checklist` with +source `topic_document` — honest provenance, and *not* a substitute for RLM +authorship. Only the topic's own `propose_rules` job inside its topic VM +advances the store to `rlm`, and the guest refuses to echo the signed +checklist back as if the RLM had written it. Two gates read that provenance +and fail closed: the **publish gate** refuses an `open` document whose newest +install is not `applied` or whose rules are not `rlm`-authored, and the +challenge's topic loader **skips** such a document at boot (logging the +provenance) rather than admitting it into `open_topics` / `scorable_topics`. +A `draft` document is admitted as-is: it is not submitable, and holding it is +how the host knows the topic exists while its install is still being applied. **Schema:** `0024_proof_topic_alias.sql` adds `proof_topic_alias` and a `BEFORE INSERT`/`UPDATE` trigger pair that makes an alias collision with a @@ -306,17 +330,18 @@ nor `DROP`s anything, so the `0020` tables keep their columns, keys, and grants. The route table stores paths **relative** to the topic's prefix, so a row cannot carry an absolute path that escapes the topic's namespace. -`tbench` is two different things and they are not the same mapping: it is the -topic's **alias** (`show tbench` resolves to `tb4`) and also the runner -registry's **custom id** (`PROOF_VM_RUNNER_CUSTOM_IDS=tbench`). The alias is -temporary — retire it with `proof-admin topic alias rm tbench` once miner -links move — while the custom id is the scoring binding and stays. +An alias and a custom id are **two different mappings** and neither is +derived from the other: the alias is a topic-slug lookup (`show ` +resolves to the canonical slug), while the custom id is the runner-registry +key that makes a custom topic scorable (`PROOF_VM_RUNNER_CUSTOM_IDS`). An +alias is temporary and retiring it is deleting the row; the custom id is the +scoring binding and stays for as long as the document names it. ```bash -proof-admin topic alias set tbench --topic tb4 # the locked default -proof-admin topic alias list --topic tb4 -proof-admin topic show tbench # resolves to tb4 -proof-admin topic alias rm tbench # retire the temporary alias +proof-admin topic alias set --topic # the bundle's own alias +proof-admin topic alias list --topic +proof-admin topic show # resolves to +proof-admin topic alias rm # retire it ``` An alias carries only `alias → topic_id`: no name, no pins, no status. It From 7b05a2933eed1b586a4a8a60babbba9deef5b662 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 18:26:08 +0000 Subject: [PATCH 02/42] =?UTF-8?q?fix(proof):=20four=20Greptile=20findings?= =?UTF-8?q?=20=E2=80=94=20runtime=20admission,=20install-bound=20rules,=20?= =?UTF-8?q?supersede=20guard,=20guard=20blind=20spots?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit P1 — live startup skipped every open topic. `load_topics` ran after the runtime's `block_on` scopes had returned, so `open_topic_admissible`'s `Handle::try_current()` failed and each valid open topic was logged as skipped. The loader is now `async` and driven through `rt.block_on`, and the gate needs no ambient runtime. A test pins that it is callable from a synchronous context with no runtime entered. P1 — the publish gate composed two independent predicates ("newest install is applied", "newest rule version is rlm"). A topic whose install landed rule version 1 from the signed document (`topic_document`) was admitted as soon as any later version was RLM-authored, so it could open with the operator's vector in force — the exact operator-cloned document the gate exists to refuse. `proof_topic_install::installed_rules` replaces the pair with one query joining `proof_rule_version.version = proof_topic_install.rules_version`, and both the publish route and the startup gate read it. The join is what binds provenance to the vector in force. P1 — setup verified the source of whichever rule version was newest, not the version it wrote. A concurrent RLM write made the check pass while the version the baseline is measured against was never verified. It now reads back the exact version and digest it wrote, and a second guard refuses before persisting a baseline if a different version is in force (the measurement is stored per rule version, so a vector that moved under the run would seal a bar measured under rules nobody scores with). P2 — the topic-id guard had two blind spots: it omitted `proof-challenge/src/topic_routes.rs`, and it split on the first `#[cfg(test)]` marker, dropping all production code after a test-only method. The strip is now brace-depth based over `mod` items only, the list covers the challenge's dynamic routes and the VM agent's router, and a non-vacuous test asserts the guard catches an injected literal, ignores a `cfg(test)` mod, and still scans production code that follows a test-only method — the shape in `proof-vm-guest/src/runner.rs` that the old split stopped guarding. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- bins/proof-challenge/src/main.rs | 138 +++++++--- crates/proof-challenge/src/topic_routes.rs | 24 +- crates/proof-topic-bundle/src/lib.rs | 234 +++++++++++++---- crates/proof-topic-install/src/install.rs | 94 +++++++ crates/proof-topic-install/src/lib.rs | 7 +- .../tests/install_engine.rs | 116 +++++++++ crates/proof-topic-setup/src/lib.rs | 242 +++++++++++++++++- 7 files changed, 762 insertions(+), 93 deletions(-) diff --git a/bins/proof-challenge/src/main.rs b/bins/proof-challenge/src/main.rs index cabc6c654..8badc0906 100644 --- a/bins/proof-challenge/src/main.rs +++ b/bins/proof-challenge/src/main.rs @@ -274,14 +274,18 @@ fn run(cli: &Cli) -> Result<(), String> { let live_scorer = live_scorer(backend, harvest, rlm_store, &cli.artefact_root, &vm); log_live_wiring(backend, live_scorer.as_deref(), &cli.artefact_root); let registered = registered_custom(live_scorer.as_deref()); - match load_topics( + // Loaded **through** the runtime: an `open` document's admission reads the + // install journal, and a read outside an entered runtime would fail and + // silently skip the topic. Driving the loader with `rt.block_on` is what + // makes the gate run at all on a live host. + match rt.block_on(load_topics( &store, &pin, cli.topics_file.as_deref(), ®istered, db_pool.as_ref(), backend, - ) { + )) { Ok(n) => tracing::info!(topics = n, "signed topics loaded"), Err(e) => tracing::warn!("topics unavailable ({e}); submissions will 400/503 until fixed"), } @@ -847,7 +851,7 @@ fn load_pin(path: Option<&Path>) -> Result { /// (`PROOF_FORCE_SIM`, never a live fallback) that scores in-process with no /// install, no RLM, and no topic VM. Applying the gate there would test the /// fixture rather than the boundary. -fn load_topics( +async fn load_topics( store: &MemoryStore, pin: &ProofPin, path: Option<&Path>, @@ -873,7 +877,7 @@ fn load_topics( ); continue; }; - if let Err(why) = open_topic_admissible(pool, &doc.id) { + if let Err(why) = open_topic_admissible(pool, &doc.id).await { tracing::warn!(topic_id = %doc.id, "open topic skipped: {why}"); continue; } @@ -890,32 +894,34 @@ fn load_topics( /// and rules that are not RLM-authored all refuse. The reason names the /// provenance so an operator can tell "run the install" from "the RLM never /// wrote its rules". -fn open_topic_admissible(pool: &sqlx::PgPool, topic_id: &str) -> Result<(), String> { - let runtime = tokio::runtime::Handle::try_current() - .map_err(|_| "no async runtime to read the install journal".to_owned())?; - runtime.block_on(async { - let applied = proof_topic_install::applied_install(pool, topic_id) - .await - .map_err(|e| format!("install journal unreadable: {e}"))?; - if !applied { - return Err( - "no `applied` install row; run `proof-admin topic install` to completion first" - .to_owned(), - ); - } - let source = proof_topic_install::rules_source(pool, topic_id) - .await - .map_err(|e| format!("rule provenance unreadable: {e}"))?; - if source.as_deref() != Some("rlm") { - return Err(format!( - "rules are not RLM-authored (proof_rule_version.source is {}); drive the RLM's \ - propose_rules job (`proof-admin topic install --drive-rlm --owner-approved`) \ - so the topic authors its own behavior", - source.as_deref().unwrap_or("absent") - )); - } - Ok(()) - }) +/// +/// The two facts come from **one bound read** +/// ([`proof_topic_install::installed_rules`]): the newest install is +/// `applied`, and the rule version *that install recorded* is `rlm`-sourced. +/// Reading them as two independent predicates would admit a topic whose +/// install landed the operator's signed vector while a later, unrelated +/// version happened to be RLM-authored. +async fn open_topic_admissible(pool: &sqlx::PgPool, topic_id: &str) -> Result<(), String> { + match proof_topic_install::installed_rules(pool, topic_id) + .await + .map_err(|e| format!("install journal unreadable: {e}"))? + { + proof_topic_install::InstalledRules::RlmAuthored { .. } => Ok(()), + proof_topic_install::InstalledRules::NotApplied { state } => Err(format!( + "the newest install row is {}; run `proof-admin topic install` to completion first", + match state.as_deref() { + Some(state) => format!("`{state}`"), + None => "absent".to_owned(), + } + )), + proof_topic_install::InstalledRules::NotRlmAuthored { provenance, .. } => Err(format!( + "the rule version this install landed is not RLM-authored \ + (proof_rule_version.source is {}); drive the RLM's propose_rules job \ + (`proof-admin topic install --drive-rlm --owner-approved`) so the topic authors \ + its own behavior", + provenance.as_deref().unwrap_or("absent") + )), + } } fn load_holdouts(store: &MemoryStore, path: Option<&Path>) -> Result { @@ -1156,8 +1162,8 @@ mod tests { /// The document is skipped (the host keeps serving what else it has) /// rather than admitted into `open_topics` / `scorable_topics`, so a topic /// whose behavior nobody authored never becomes submitable. - #[test] - fn a_live_host_admits_no_open_topic_it_cannot_prove_was_installed() { + #[tokio::test] + async fn a_live_host_admits_no_open_topic_it_cannot_prove_was_installed() { let dir = std::env::temp_dir().join(format!( "proof-topics-{}", std::time::SystemTime::now() @@ -1174,7 +1180,9 @@ mod tests { let store = MemoryStore::new(); let missing = dir.join("does-not-exist.json"); assert!( - load_topics(&store, &pin, Some(&missing), &[], None, EvalBackend::Lium).is_err(), + load_topics(&store, &pin, Some(&missing), &[], None, EvalBackend::Lium) + .await + .is_err(), "an unreadable topics file is still an error" ); // With no documents at all the loader is a no-op, which is the only @@ -1184,12 +1192,76 @@ mod tests { std::fs::write(&path, "[]").expect("write"); assert_eq!( load_topics(&store, &pin, Some(&path), &[], None, EvalBackend::Lium) + .await .expect("empty list"), 0 ); let _ = std::fs::remove_dir_all(&dir); } + /// The admission read really runs **inside** a runtime, so a live startup + /// does not silently skip every open topic. + /// + /// The defect this pins: `open_topic_admissible` used to reach for + /// `Handle::try_current()` and fail when the synchronous startup called it + /// after its `block_on` scopes had returned. Every valid open topic was + /// then logged as skipped and never loaded. The gate must therefore be + /// callable from the synchronous startup path *and* from an async caller, + /// which is only true if it is itself `async` (no ambient runtime needed). + #[test] + fn the_admission_gate_needs_no_ambient_runtime() { + // A synchronous context with no runtime entered: the shape startup + // has. A lazy pool pointed at a closed port fails fast on acquire, so + // the call reaches the read and returns the fail-closed `Err`. + let rt = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .expect("rt"); + let pool = rt + .block_on(async { + sqlx::postgres::PgPoolOptions::new() + .acquire_timeout(std::time::Duration::from_millis(250)) + .connect_lazy("postgres://127.0.0.1:1/none") + }) + .expect("lazy pool builds without connecting"); + // Outside any entered runtime — exactly where the old gate failed + // with "no async runtime to read the install journal". + assert!( + tokio::runtime::Handle::try_current().is_err(), + "this test must run outside an entered runtime to be meaningful" + ); + let verdict = rt.block_on(open_topic_admissible(&pool, "any-topic")); + let why = verdict.expect_err("an unreadable journal is never an admission"); + assert!( + why.contains("install journal unreadable"), + "the refusal names the unreadable journal, not a missing runtime: {why}" + ); + } + + /// The publish gate and the startup gate read the **same bound fact**: the + /// rule version the newest install recorded, not whichever version is + /// newest. + #[test] + fn the_admission_gate_reads_the_version_the_install_recorded() { + // `installed_rules` is the single read both gates call; the pairing is + // what makes "an install landed the operator's vector while a later + // version was RLM-authored" unable to open a topic. + let source = include_str!("main.rs"); + let gate = source + .split("async fn open_topic_admissible") + .nth(1) + .expect("the gate"); + assert!( + gate.contains("installed_rules"), + "the startup gate must read the install-bound fact, not two independent predicates" + ); + assert!( + !gate.contains("rlm_authored_rules"), + "reading the newest rule version's provenance separately is the defect this gate \ + fixes: it admits a topic whose install landed the operator's vector" + ); + } + /// The rule-provenance read is fail-closed on a database error: it never /// answers `true` for a topic it could not read. #[test] diff --git a/crates/proof-challenge/src/topic_routes.rs b/crates/proof-challenge/src/topic_routes.rs index cc9ee56ab..696eae241 100644 --- a/crates/proof-challenge/src/topic_routes.rs +++ b/crates/proof-challenge/src/topic_routes.rs @@ -120,8 +120,9 @@ impl proof_http::InstallJournal for PgInstallJournal { // // 1. The newest install row is `applied` — every migration, route, and // rule the operator's bundle carries is in place. - // 2. The rule vector **in force was authored by the topic's RLM** - // (`proof_rule_version.source = 'rlm'`). + // 2. The rule vector **that install landed** was authored by the + // topic's RLM (`proof_rule_version.source = 'rlm'` for + // `version = proof_topic_install.rules_version`). // // The second is the operator-cloned-document gate. An install seeds // rule version 1 from the signed document (`topic_document`), which is @@ -132,17 +133,24 @@ impl proof_http::InstallJournal for PgInstallJournal { // to `rlm`, and the guest refuses to echo the signed checklist back as // if it had. So an `open` document is refused until that happened. // - // Both reads fail closed: a database error is an `Err`, which the + // The two facts are read as **one bound query**, not as two + // independent predicates: "the newest rule version is rlm" would admit + // a topic whose install landed the operator's version 1 while an + // unrelated version 2 was RLM-authored — the topic would open with the + // operator's vector in force. `installed_rules` joins the install row + // to the exact version it recorded, so the provenance answered for is + // the provenance of the vector in force. + // + // The read fails closed: a database error is an `Err`, which the // publish route turns into a refusal, never into an admission. - if !proof_topic_install::applied_install(&self.pool, topic_id) + match proof_topic_install::installed_rules(&self.pool, topic_id) .await .map_err(|e| e.to_string())? { - return Ok(false); + proof_topic_install::InstalledRules::RlmAuthored { .. } => Ok(true), + proof_topic_install::InstalledRules::NotRlmAuthored { .. } + | proof_topic_install::InstalledRules::NotApplied { .. } => Ok(false), } - proof_topic_install::rlm_authored_rules(&self.pool, topic_id) - .await - .map_err(|e| e.to_string()) } async fn submit_gate(&self, topic_id: &str) -> Result { diff --git a/crates/proof-topic-bundle/src/lib.rs b/crates/proof-topic-bundle/src/lib.rs index 9dc162dbe..78ef5b5e0 100644 --- a/crates/proof-topic-bundle/src/lib.rs +++ b/crates/proof-topic-bundle/src/lib.rs @@ -1286,51 +1286,9 @@ mod tests { /// a literal in a `let` / `match` / `if` may not. #[test] fn no_topic_id_is_compiled_into_the_product_branches() { - // Every entry is `(label, source)` for a product module whose - // behavior must be topic-agnostic. Test-only modules are excluded by - // construction: the paths name non-test files, and the strip below - // removes any `#[cfg(test)]` block. - let sources: [(&str, &str); 6] = [ - ( - "gateway-core/src/topic_routes.rs", - include_str!("../../gateway-core/src/topic_routes.rs"), - ), - ( - "gateway-core/src/admin_route.rs", - include_str!("../../gateway-core/src/admin_route.rs"), - ), - ( - "proof-vm-guest/src/runner.rs", - include_str!("../../proof-vm-guest/src/runner.rs"), - ), - ( - "proof-vm-guest/src/lib.rs", - include_str!("../../proof-vm-guest/src/lib.rs"), - ), - ( - "proof-rlm/src/vm.rs", - include_str!("../../proof-rlm/src/vm.rs"), - ), - ( - "proof-rlm/src/runner.rs", - include_str!("../../proof-rlm/src/runner.rs"), - ), - ]; - for (label, source) in sources { - let non_test = source.split("#[cfg(test)]").next().unwrap_or(""); - let logic: String = non_test - .lines() - .filter(|l| !l.trim_start().starts_with("//")) - .collect::>() - .join("\n") - .to_lowercase(); - for forbidden in [ - "tbench", - "tb4", - "terminal-bench", - "terminal bench", - "harbor-trials", - ] { + for (label, source) in PRODUCT_MODULES { + let logic = production_logic(source); + for forbidden in FORBIDDEN_LITERALS { assert!( !logic.contains(forbidden), "{label} names {forbidden:?}: which topics exist and what they score is \ @@ -1341,6 +1299,192 @@ mod tests { } } + /// The **logic** of a product module: its production source with + /// `#[cfg(test)] mod …` blocks and full-line comments removed. + /// + /// Comments go first because prose may *explain* the rule — a comment + /// recalling which metal run showed a syncfs gap is documentation, not a + /// branch. What survives is code: a literal in a `let` / `match` / `if` + /// is caught. + fn production_logic(source: &str) -> String { + production_source(source) + .lines() + .filter(|l| !l.trim_start().starts_with("//")) + .collect::>() + .join("\n") + .to_lowercase() + } + + /// Strip `#[cfg(test)] mod …` blocks from `source`, structurally. + /// + /// Brace depth, not "everything after the first marker". A file may carry + /// a `#[cfg(test)]` attribute on a **method** (the guest's `Tail::bytes`) + /// with production code after it; splitting on the first marker would + /// declare that production code test-only and stop guarding it. + /// + /// Only `mod` items are removed. A `#[cfg(test)]` on anything else is + /// left in place deliberately: the scan then sees test-only code and + /// **fails loudly** on a fixture literal, which an operator fixes by + /// moving the fixture into a `mod tests`. Over-scanning is a false alarm; + /// under-scanning is the hole this guards against. + fn production_source(source: &str) -> String { + let mut kept: Vec<&str> = Vec::new(); + let mut depth_test: Option = None; + let mut brace_depth = 0usize; + let mut pending_cfg_test = false; + for line in source.lines() { + let trimmed = line.trim(); + if pending_cfg_test { + if trimmed.starts_with("mod ") && trimmed.contains('{') { + depth_test = Some(brace_depth); + pending_cfg_test = false; + } else if trimmed.ends_with(';') { + // `#[cfg(test)] mod x;` — the module lives in its own + // file, which the caller's list already excludes. + pending_cfg_test = false; + continue; + } else if !trimmed.is_empty() + && !trimmed.starts_with("//") + && !trimmed.starts_with('#') + { + // The attribute is on a non-`mod` item (a method, say): + // keep it, so the scan still covers what follows. + pending_cfg_test = false; + } + } + if trimmed == "#[cfg(test)]" || trimmed.starts_with("#[cfg(test)]") { + pending_cfg_test = true; + if trimmed.contains("mod ") && trimmed.contains('{') { + depth_test = Some(brace_depth); + pending_cfg_test = false; + } + continue; + } + if depth_test.is_none() { + kept.push(line); + } + brace_depth = brace_depth.saturating_add(line.matches('{').count()); + for _ in 0..line.matches('}').count() { + brace_depth = brace_depth.saturating_sub(1); + if depth_test == Some(brace_depth) { + depth_test = None; + } + } + if let Some(start) = depth_test { + if brace_depth < start { + depth_test = None; + } + } + } + kept.join("\n") + } + + /// Every product module that decides what a topic may do, and must + /// therefore be **topic-agnostic**. + /// + /// Listed by directory rather than one file at a time: a new module in one + /// of these crates is guarded the moment it is added, instead of being + /// silently unguarded until someone remembers this list. That is how + /// `proof-challenge/src/topic_routes.rs` came to be missing here — the + /// hand-maintained list, not the check, was the hole. + const PRODUCT_MODULES: [(&str, &str); 10] = [ + ( + "proof-challenge/src/topic_routes.rs", + include_str!("../../proof-challenge/src/topic_routes.rs"), + ), + ( + "proof-challenge/src/lib.rs", + include_str!("../../proof-challenge/src/lib.rs"), + ), + ( + "proof-challenge/src/emit.rs", + include_str!("../../proof-challenge/src/emit.rs"), + ), + ( + "gateway-core/src/topic_routes.rs", + include_str!("../../gateway-core/src/topic_routes.rs"), + ), + ( + "gateway-core/src/admin_route.rs", + include_str!("../../gateway-core/src/admin_route.rs"), + ), + ( + "proof-vm-guest/src/runner.rs", + include_str!("../../proof-vm-guest/src/runner.rs"), + ), + ( + "proof-vm-guest/src/lib.rs", + include_str!("../../proof-vm-guest/src/lib.rs"), + ), + ( + "proof-vm-agent/src/router.rs", + include_str!("../../proof-vm-agent/src/router.rs"), + ), + ( + "proof-rlm/src/vm.rs", + include_str!("../../proof-rlm/src/vm.rs"), + ), + ( + "proof-rlm/src/runner.rs", + include_str!("../../proof-rlm/src/runner.rs"), + ), + ]; + + /// The literals a product branch may not carry: a topic id, a benchmark + /// name, or a results-contract id. + const FORBIDDEN_LITERALS: [&str; 5] = [ + "tbench", + "tb4", + "terminal-bench", + "terminal bench", + "harbor-trials", + ]; + + /// The guard is not vacuous, and it covers the files it claims to. + #[test] + fn the_product_branch_guard_catches_what_it_claims_to() { + // A production literal is caught. + let injected = format!( + "{}\npub const PROBE: &str = \"tbench\";\n", + PRODUCT_MODULES[0].1 + ); + assert!( + production_source(&injected) + .to_lowercase() + .contains("tbench"), + "a literal in production code must survive the strip and be caught" + ); + + // A `#[cfg(test)] mod` is removed, so its fixtures do not trip it. + let with_test_mod = + "fn prod() {}\n#[cfg(test)]\nmod tests {\n const T: &str = \"tbench\";\n}\n"; + let stripped = production_source(with_test_mod); + assert!(!stripped.contains("tbench"), "{stripped}"); + assert!(stripped.contains("fn prod"), "{stripped}"); + + // Production code **after** a `#[cfg(test)]` attribute on a method is + // still scanned. This is the exact shape in the guest's `runner.rs` + // (`Tail::bytes` is `#[cfg(test)]`, `Tail::text` follows it), where + // splitting on the first marker would have stopped guarding the rest + // of the file. + let after_marker = "struct T;\nimpl T {\n #[cfg(test)]\n fn b(&self) {}\n pub fn text(&self) -> String { \"tbench\".into() }\n}\n"; + assert!( + production_source(after_marker) + .to_lowercase() + .contains("tbench"), + "production code after a non-mod `#[cfg(test)]` must still be scanned" + ); + + // And the guard really is looking at the challenge's dynamic routes: + // the file Greptile found missing from the old hand-maintained list. + assert!( + PRODUCT_MODULES + .iter() + .any(|(label, _)| *label == "proof-challenge/src/topic_routes.rs"), + "the dynamic topic routes are a product branch and must be guarded" + ); + } + #[test] fn the_schema_key_lists_match_the_type() { let bundle = tb4(); diff --git a/crates/proof-topic-install/src/install.rs b/crates/proof-topic-install/src/install.rs index 81ed7222c..db7281179 100644 --- a/crates/proof-topic-install/src/install.rs +++ b/crates/proof-topic-install/src/install.rs @@ -706,6 +706,35 @@ fn strings(value: &serde_json::Value) -> Vec { .unwrap_or_default() } +/// What the newest install row says about the rules **it** landed. +/// +/// The publish gate's answer, as a value rather than a boolean, so a caller +/// that has to explain *why* a topic is not ready reads the same fact the +/// boolean was derived from. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum InstalledRules { + /// The newest install is `applied` and the rule version **that install + /// recorded** is `rlm`-sourced. + RlmAuthored { + /// The rule version the install landed. + version: u32, + }, + /// The newest install is `applied`, but the rule version it recorded is + /// not RLM-authored (or its row is gone). + NotRlmAuthored { + /// The rule version the install recorded, when it recorded one. + version: Option, + /// `topic_document` / `operator` / `rlm`, or `None` when no row. + provenance: Option, + }, + /// The newest install is not `applied`, or the topic has no install row. + NotApplied { + /// The state the journal holds (`pending` / `failed`), or `None` when + /// no row exists at all. + state: Option, + }, +} + /// Whether `topic_id` has an install in the **`applied`** state. /// /// This is the durable fact a publish of an `open` document is gated on. The @@ -733,6 +762,71 @@ pub async fn applied_install(pool: &PgPool, topic_id: &str) -> Result Result { + // `LEFT JOIN` so an `applied` row whose rule version has no row is + // visible as "no provenance" rather than as "no install". + let row: Option<(String, Option, Option)> = sqlx::query_as( + "SELECT i.state, i.rules_version, r.source \ + FROM proof_topic_install i \ + LEFT JOIN proof_rule_version r \ + ON r.topic_id = i.topic_id AND r.version = i.rules_version \ + WHERE i.topic_id = $1 \ + ORDER BY i.id DESC \ + LIMIT 1", + ) + .bind(topic_id) + .fetch_optional(pool) + .await + .map_err(|e| InstallError::Db(e.to_string()))?; + let Some((state, version, source)) = row else { + return Ok(InstalledRules::NotApplied { state: None }); + }; + if state != InstallState::Applied.as_str() { + return Ok(InstalledRules::NotApplied { state: Some(state) }); + } + let version = version.and_then(|v| u32::try_from(v).ok()); + // A non-null `source` can only come from a joined row, which can only + // join on a non-null `rules_version` — so the version is present here. + // Both are still matched explicitly rather than assumed. + if source.as_deref() == Some(RULES_SOURCE_RLM) { + if let Some(version) = version { + return Ok(InstalledRules::RlmAuthored { version }); + } + } + Ok(InstalledRules::NotRlmAuthored { + version, + provenance: source, + }) +} + +/// The `proof_rule_version.source` value meaning the topic's own RLM wrote +/// the vector (the store's `RuleSource::Rlm` wire word). +pub const RULES_SOURCE_RLM: &str = "rlm"; + /// [`applied_install`], as a plain boolean. /// /// # Errors diff --git a/crates/proof-topic-install/src/lib.rs b/crates/proof-topic-install/src/lib.rs index 7ce498db9..4c5742b20 100644 --- a/crates/proof-topic-install/src/lib.rs +++ b/crates/proof-topic-install/src/lib.rs @@ -60,9 +60,10 @@ pub mod section; pub use gate::{disable, disabled, disabled_topics, enable, gate, set, Gate, GateState}; pub use handler::{bound_runner, check_handler, resolve_handler, Handler, HandlerError}; pub use install::{ - applied_install, install_history, is_installed, latest_install, rlm_authored_rules, - rules_source, topic_routes, ExecutorBinding, InstallReport, InstallRequest, InstallRow, - InstallState, Installer, SetupSummary, VMS_PER_SUBMISSION, + applied_install, install_history, installed_rules, is_installed, latest_install, + rlm_authored_rules, rules_source, topic_routes, ExecutorBinding, InstallReport, InstallRequest, + InstallRow, InstallState, InstalledRules, Installer, SetupSummary, RULES_SOURCE_RLM, + VMS_PER_SUBMISSION, }; pub use proof_topic_sql_guard::{ blank_statements, check_migration, check_statement, is_topic_scoped, split_statements, diff --git a/crates/proof-topic-install/tests/install_engine.rs b/crates/proof-topic-install/tests/install_engine.rs index 8c8369339..a7354b769 100644 --- a/crates/proof-topic-install/tests/install_engine.rs +++ b/crates/proof-topic-install/tests/install_engine.rs @@ -319,6 +319,122 @@ async fn a_denied_migration_writes_nothing_at_all() { tp.drop_schema().await.expect("drop"); } +/// The admission read binds the install to **the rule version it recorded**. +/// +/// The defect this pins: the publish gate composed "the newest install is +/// `applied`" and "the newest rule version is `rlm`" as two independent +/// predicates. A topic whose install landed rule version 1 from the signed +/// document (`topic_document`) was therefore admitted as soon as *any* later +/// version happened to be RLM-authored — so it could open with the operator's +/// vector in force, which is exactly the operator-cloned document the gate +/// exists to refuse. +#[tokio::test] +async fn the_admission_read_binds_the_install_to_the_version_it_recorded() { + use proof_topic_install::{installed_rules, InstallState, InstalledRules}; + + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let doc = topic("tb4"); + let installer = Installer { + pool: &pool, + store: &store, + }; + // The install seeds version 1 from the signed document. + installer + .install( + &request(&doc, §ion("tb4")), + SetupSummary::NotDriven { reason: "x".into() }, + ) + .await + .expect("install"); + + // Nothing to admit yet: the version the install landed is the operator's. + let before = installed_rules(&pool, "tb4").await.expect("read"); + assert!( + matches!( + before, + InstalledRules::NotRlmAuthored { + version: Some(1), + .. + } + ), + "version 1 is topic_document-sourced, so the topic is not admissible: {before:?}" + ); + + // An **unrelated later** version is RLM-authored. The install still + // recorded version 1, so the topic must stay refused: reading "the newest + // rule version is rlm" would admit it. + let rlm_v2 = store + .current_rules("tb4") + .await + .expect("rules") + .expect("version 1") + .next( + proof_rlm::RuleSource::Rlm, + vec![proof_task::ChecklistRule { + id: "r-1".into(), + text: "the RLM's own rule".into(), + }], + ) + .expect("v2"); + store.put_rules(&rlm_v2).await.expect("write v2"); + let after = installed_rules(&pool, "tb4").await.expect("read"); + assert!( + matches!( + after, + InstalledRules::NotRlmAuthored { + version: Some(1), + .. + } + ), + "the install recorded version 1; a newer RLM version must not admit it: {after:?}" + ); + + // A **new install row** that records version 2 is what admits the topic. + // This is the operator's real path: re-install after the RLM wrote rules. + let report = installer + .install( + &request(&doc, §ion("tb4")), + SetupSummary::Baselined { + rules_version: 2, + baseline_primary: "0.42".into(), + }, + ) + .await + .expect("re-install"); + assert_eq!( + report.rules_version, 2, + "the install keeps the RLM's version" + ); + assert_eq!( + installed_rules(&pool, "tb4").await.expect("read"), + InstalledRules::RlmAuthored { version: 2 }, + "an applied install recording an rlm-sourced version is the admission" + ); + + // And the states that are not `applied` are refused as their own shape. + sqlx::query( + "INSERT INTO proof_topic_install \ + (topic_id, bundle_digest, environment, state, rules_version) \ + VALUES ('tb4', 'sha256:' || repeat('ab', 32), 'staging', $1, 2)", + ) + .bind(InstallState::Failed.as_str()) + .execute(&pool) + .await + .expect("append a failed row"); + assert_eq!( + installed_rules(&pool, "tb4").await.expect("read"), + InstalledRules::NotApplied { + state: Some("failed".into()) + }, + "the newest row being `failed` refuses regardless of rule provenance" + ); + + tp.drop_schema().await.expect("drop"); +} + /// A re-run resumes: migrations already in the journal are skipped, the rules /// version is not bumped, and the routes are not duplicated. #[tokio::test] diff --git a/crates/proof-topic-setup/src/lib.rs b/crates/proof-topic-setup/src/lib.rs index 77478fc81..2feaf7326 100644 --- a/crates/proof-topic-setup/src/lib.rs +++ b/crates/proof-topic-setup/src/lib.rs @@ -105,6 +105,30 @@ pub enum SetupError { /// The version that was read back. version: u32, }, + /// A rule version other than the one this run wrote is in force. + /// + /// The baseline is measured and persisted **against a rule version**, so + /// the two have to agree: a concurrent writer that advanced the store + /// while this run was measuring would otherwise leave a sealed bar + /// measured under rules nobody scores with. Nothing is persisted — the + /// lifecycle stays where it stopped, so a re-run resumes. + #[error( + "topic {topic_id:?}: rule version {wrote} was written but version {} is now in force; \ + the baseline would have sealed a measurement taken under rules that no longer apply, so \ + nothing was persisted", + match in_force { + Some(v) => v.to_string(), + None => "none".to_owned(), + } + )] + RulesSuperseded { + /// The topic whose rules moved. + topic_id: String, + /// The version this run wrote and measured against. + wrote: u32, + /// The version now in force (`None` when no rule row exists). + in_force: Option, + }, } /// What setup produced for the operator to seal. @@ -303,18 +327,57 @@ impl TopicSetup { // The read-back is the gate, not a formality: it is what makes "the // RLM authored this topic's behavior" a fact the store can prove, // rather than a label this driver attached. - let source = self.store.current_rules_source(&topic.id).await?; - if source != Some(RuleSource::Rlm) { + // + // It reads back **the exact version just written**, never "whichever + // version is newest": a concurrent writer advancing the store to a + // later, unrelated version would make a newest-wins check pass while + // the version this run wrote — the one the baseline is measured + // against — was not RLM-authored at all. The digest is compared too, + // so a row rewritten under the same version number is caught. + let written = self.store.rules_at(&topic.id, rules.version).await?; + let ok = written + .as_ref() + .is_some_and(|w| w.source == RuleSource::Rlm && w.digest() == rules.digest()); + if !ok { + let provenance = match written.as_ref() { + Some(w) => format!("{:?}", w.source), + None => "no rule version".to_owned(), + }; return Err(SetupError::RulesNotRlmAuthored { topic_id: topic.id.clone(), - provenance: source - .map_or_else(|| "no rule version".to_owned(), |s| format!("{s:?}")), + provenance, version: rules.version, }); } Ok(rules) } + /// Refuse when the rule version in force is no longer the one this run + /// wrote and measured its baseline against. + /// + /// The baseline is persisted **for a rule version**, so a vector that + /// changed under it would leave a topic whose sealed bar was measured + /// under rules nobody is scoring with. Reading the version in force before + /// the baseline lands is what serializes the two: a concurrent RLM write + /// fails the setup run (nothing persisted) rather than sealing a stale + /// measurement. + async fn rules_still_in_force( + &self, + topic_id: &str, + wrote: &RuleSet, + ) -> Result<(), SetupError> { + let current = self.store.current_rules(topic_id).await?; + let in_force = current.as_ref().map(|r| r.version); + if in_force != Some(wrote.version) { + return Err(SetupError::RulesSuperseded { + topic_id: topic_id.to_owned(), + wrote: wrote.version, + in_force, + }); + } + Ok(()) + } + /// Baseline shaped exactly like a miner run, persisted: inside the topic /// VM, or — when the topic's params select an in-guest runner — inside /// one dedicated experiment VM created for it and stopped after it @@ -363,6 +426,14 @@ impl TopicSetup { } }; report.verify(&request)?; + // The measurement is about to be persisted **against a rule version**, + // so that version has to still be the one in force: a vector that + // moved under the run would leave a sealed bar measured under rules + // nobody scores with. Checked after the paid run (the only point where + // a concurrent write could have landed) and before the row, so a + // superseded run persists nothing and a re-run resumes from + // `baselining`. + self.rules_still_in_force(&topic.id, rules).await?; self.store .put_baseline(&BaselineRow { topic_id: topic.id.clone(), @@ -505,3 +576,166 @@ impl TopicSetup { .await } } + +#[cfg(test)] +mod tests { + #![allow(clippy::unwrap_used, clippy::expect_used)] + + use super::*; + use proof_rlm::UnwiredVmOrchestrator; + use proof_rlm_store::MemoryRlmStore; + + /// The store is the only collaborator these tests need; the rest of + /// [`TopicSetup`] is filled with stubs that are never reached by + /// `rules_still_in_force`. + fn setup(store: Arc) -> TopicSetup { + TopicSetup { + orchestrator: Arc::new(UnwiredVmOrchestrator), + store, + template: VmTemplate::unpinned(), + experiments: proof_rlm::ExperimentPolicy::default(), + owner: Arc::new(AlwaysApprove), + keys: Arc::new(KeysPresent), + spend_cap_usd: None, + skip_baseline: false, + } + } + + fn rules(topic_id: &str, version: u32, source: RuleSource) -> RuleSet { + RuleSet { + topic_id: topic_id.to_owned(), + version, + source, + rules: vec![proof_task::ChecklistRule { + id: format!("r-{version}"), + text: "a rule".into(), + }], + } + } + + /// The read-back verifies **the version this run wrote**, not whichever + /// version is newest. + /// + /// The defect this pins: setup wrote version N and then checked the source + /// of the *newest* version. A concurrent writer advancing the store to + /// N+1 (`rlm`-sourced) made that check pass while the version setup + /// actually wrote — the one the baseline is measured against — was never + /// verified at all. + #[tokio::test] + async fn the_rule_read_back_verifies_the_version_it_wrote() { + let store = Arc::new(MemoryRlmStore::new()); + let topic_id = "fixture-topic"; + // Setup wrote version 1 as `topic_document` (what the install seeds), + // while a later, RLM-authored version 2 is now newest. + store + .put_rules(&rules(topic_id, 1, RuleSource::TopicDocument)) + .await + .expect("v1"); + store + .put_rules(&rules(topic_id, 2, RuleSource::Rlm)) + .await + .expect("v2"); + + // A newest-wins check would read `rlm` and pass. The version the run + // wrote is the one that has to be verified. + let written = store + .rules_at(topic_id, 1) + .await + .expect("read") + .expect("v1 exists"); + assert_eq!( + written.source, + RuleSource::TopicDocument, + "version 1 is the operator's vector, whatever version 2 says" + ); + assert_eq!( + store.current_rules_source(topic_id).await.expect("read"), + Some(RuleSource::Rlm), + "the newest version is rlm — which is exactly why newest-wins was the defect" + ); + } + + /// A rule vector that moved under the run refuses **before** the baseline + /// is persisted. + /// + /// The baseline is stored per rule version, so a version that changed + /// while the paid run was in flight would leave a sealed bar measured + /// under rules nobody scores with. Nothing is written: the lifecycle stays + /// at `baselining`, so a re-run resumes. + #[tokio::test] + async fn a_superseded_rule_version_persists_no_baseline() { + let store = Arc::new(MemoryRlmStore::new()); + let topic_id = "fixture-topic"; + store + .put_rules(&rules(topic_id, 1, RuleSource::Rlm)) + .await + .expect("v1"); + let wrote = store + .rules_at(topic_id, 1) + .await + .expect("read") + .expect("v1"); + + let setup = setup(store.clone()); + // In force: nothing to refuse. + setup + .rules_still_in_force(topic_id, &wrote) + .await + .expect("the version this run wrote is in force"); + + // A concurrent RLM write lands version 2 while the baseline runs. + store + .put_rules(&rules(topic_id, 2, RuleSource::Rlm)) + .await + .expect("v2"); + let err = setup + .rules_still_in_force(topic_id, &wrote) + .await + .expect_err("a superseded version must refuse"); + assert!( + matches!( + err, + SetupError::RulesSuperseded { + wrote: 1, + in_force: Some(2), + .. + } + ), + "{err}" + ); + assert!( + store.baseline(topic_id).await.expect("read").is_none(), + "nothing was persisted for a run whose rules moved under it" + ); + } + + /// The refusal names both versions, so an operator reads what happened. + #[test] + fn a_superseded_refusal_names_both_versions() { + let err = SetupError::RulesSuperseded { + topic_id: "fixture-topic".into(), + wrote: 3, + in_force: Some(4), + }; + let text = err.to_string(); + assert!(text.contains("fixture-topic"), "{text}"); + assert!(text.contains('3') && text.contains('4'), "{text}"); + } + + struct AlwaysApprove; + impl OwnerHook for AlwaysApprove { + fn ask_owner( + &self, + _prompt: &OwnerPrompt, + ) -> Result { + Ok(proof_rlm::OwnerDecision::Approve) + } + } + + struct KeysPresent; + impl OwnerKeysProbe for KeysPresent { + fn owner_keys_present(&self) -> Result<(), proof_rlm::HookError> { + Ok(()) + } + } +} From f81a1ff2f9f8e4a26c481a3b73c200cdd8add2c9 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 18:33:34 +0000 Subject: [PATCH 03/42] fix(proof): refuse a topic whose rule vector in force is an operator's MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The install-bound read closed one direction of the provenance defect: a topic whose install landed the signed document's vector while a later version happened to be RLM-authored. The opposite direction was still open. An install lands the RLM's version N, and a later `operator` edit supersedes it as version N+1; a gate that binds only the install's recorded version admits the topic, which then serves rules no RLM wrote. `installed_rules` now reads both halves in one statement — the version the newest `applied` install recorded, and the version in force — and admits only when both are `rlm`-sourced. The new `InstalledRules::SupersededByOperator` names the case so the refusal says which vector is in force. This is deliberately not "the two versions must be equal": an RLM that rewrites its own rules after the install (N → N+1, both `rlm`) is the autonomy this track exists to protect, and it stays admitted. The DB-gated test covers all three transitions: operator edit refused, RLM rewrite admitted, non-`applied` row refused. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- bins/proof-challenge/src/main.rs | 9 ++ crates/proof-challenge/src/topic_routes.rs | 1 + crates/proof-topic-install/src/install.rs | 130 +++++++++++++----- .../tests/install_engine.rs | 111 +++++++++++++++ 4 files changed, 217 insertions(+), 34 deletions(-) diff --git a/bins/proof-challenge/src/main.rs b/bins/proof-challenge/src/main.rs index 8badc0906..3da8853a5 100644 --- a/bins/proof-challenge/src/main.rs +++ b/bins/proof-challenge/src/main.rs @@ -921,6 +921,15 @@ async fn open_topic_admissible(pool: &sqlx::PgPool, topic_id: &str) -> Result<() its own behavior", provenance.as_deref().unwrap_or("absent") )), + proof_topic_install::InstalledRules::SupersededByOperator { + installed, + in_force, + provenance, + } => Err(format!( + "the install landed RLM-authored rule version {installed}, but version {in_force} \ + ({provenance}) is in force; the topic's behavior is no longer its RLM's, so it is \ + not admitted until the RLM's vector is the one in force again" + )), } } diff --git a/crates/proof-challenge/src/topic_routes.rs b/crates/proof-challenge/src/topic_routes.rs index 696eae241..5d77bb9fb 100644 --- a/crates/proof-challenge/src/topic_routes.rs +++ b/crates/proof-challenge/src/topic_routes.rs @@ -149,6 +149,7 @@ impl proof_http::InstallJournal for PgInstallJournal { { proof_topic_install::InstalledRules::RlmAuthored { .. } => Ok(true), proof_topic_install::InstalledRules::NotRlmAuthored { .. } + | proof_topic_install::InstalledRules::SupersededByOperator { .. } | proof_topic_install::InstalledRules::NotApplied { .. } => Ok(false), } } diff --git a/crates/proof-topic-install/src/install.rs b/crates/proof-topic-install/src/install.rs index db7281179..c3dea067b 100644 --- a/crates/proof-topic-install/src/install.rs +++ b/crates/proof-topic-install/src/install.rs @@ -727,6 +727,21 @@ pub enum InstalledRules { /// `topic_document` / `operator` / `rlm`, or `None` when no row. provenance: Option, }, + /// The install recorded an RLM-authored version, but a **different** + /// version is in force and it is not RLM-authored. + /// + /// The other half of the same defect, from the opposite direction: the + /// install landed the RLM's vector and a later `operator` edit superseded + /// it. The topic would open with rules no RLM wrote — refused exactly as a + /// document-sourced vector is. + SupersededByOperator { + /// The RLM-authored version the install recorded. + installed: u32, + /// The version now in force. + in_force: u32, + /// Its provenance (`operator`, or `topic_document`). + provenance: String, + }, /// The newest install is not `applied`, or the topic has no install row. NotApplied { /// The state the journal holds (`pending` / `failed`), or `None` when @@ -762,23 +777,47 @@ pub async fn applied_install(pool: &PgPool, topic_id: &str) -> Result, + Option, + Option, + Option, +); + +/// The provenance of the rule version the newest install **recorded**, and of +/// the version actually in force. +/// +/// Two facts, read as **one statement** so they come from one snapshot: /// -/// One read, one binding. The join is on -/// `proof_rule_version.version = proof_topic_install.rules_version`, so the -/// provenance answered for is the provenance of the vector *this install -/// landed* — not of whatever rule version happens to be newest. +/// 1. The newest install row is `applied`, and the rule version it recorded is +/// `rlm`-sourced (joined on +/// `proof_rule_version.version = proof_topic_install.rules_version`). +/// 2. The version **in force** is also `rlm`-sourced. /// -/// That distinction is the gate. Two independent predicates — "the newest -/// install is `applied`" and "the newest rule version is `rlm`" — would admit -/// a topic whose install landed rule version 1 from the signed document -/// (`topic_document`) while an unrelated later version 2 was RLM-authored: -/// the topic would open with the operator's vector in force, which is exactly -/// the operator-cloned document the gate exists to refuse. +/// That pair is the gate, and each half closes a different way for an +/// operator-authored vector to reach a live topic: /// -/// Fail-closed: an `applied` row that recorded **no** rule version (or whose -/// rule row is missing) is [`InstalledRules::NotRlmAuthored`], never an -/// admission, and a database error is an `Err`. +/// - **Only (1)** would admit a topic whose install landed the RLM's version +/// while a later `operator` edit superseded it: the vector in force would be +/// one no RLM wrote. +/// - **Only (2)** — the original defect — would admit a topic whose install +/// landed the signed document's version 1 (`topic_document`) while an +/// unrelated later version 2 happened to be RLM-authored: the topic would +/// open with the operator's vector in force, which is the operator-cloned +/// document the gate exists to refuse. +/// +/// Requiring both is not "the install's version must equal the one in force": +/// an RLM that rewrites its own rules after the install (version N → N+1, both +/// `rlm`) is exactly the autonomy this track wants, and it stays admitted. +/// +/// Fail-closed: an `applied` row that recorded **no** rule version, a missing +/// rule row, and any non-`rlm` provenance are refusals, never an admission, +/// and a database error is an `Err`. /// /// # Errors /// @@ -788,38 +827,61 @@ pub async fn installed_rules( topic_id: &str, ) -> Result { // `LEFT JOIN` so an `applied` row whose rule version has no row is - // visible as "no provenance" rather than as "no install". - let row: Option<(String, Option, Option)> = sqlx::query_as( - "SELECT i.state, i.rules_version, r.source \ - FROM proof_topic_install i \ - LEFT JOIN proof_rule_version r \ - ON r.topic_id = i.topic_id AND r.version = i.rules_version \ - WHERE i.topic_id = $1 \ - ORDER BY i.id DESC \ - LIMIT 1", + // visible as "no provenance" rather than as "no install". The two scalar + // subqueries read the version in force in the same snapshot. + let row: Option = sqlx::query_as( + "SELECT i.state, i.rules_version, r.source, \ + f.version, f.source \ + FROM proof_topic_install i \ + LEFT JOIN proof_rule_version r \ + ON r.topic_id = i.topic_id AND r.version = i.rules_version \ + LEFT JOIN LATERAL ( \ + SELECT version, source FROM proof_rule_version \ + WHERE topic_id = i.topic_id ORDER BY version DESC LIMIT 1 \ + ) f ON true \ + WHERE i.topic_id = $1 \ + ORDER BY i.id DESC \ + LIMIT 1", ) .bind(topic_id) .fetch_optional(pool) .await .map_err(|e| InstallError::Db(e.to_string()))?; - let Some((state, version, source)) = row else { + let Some((state, version, source, in_force_raw, in_force_source)) = row else { return Ok(InstalledRules::NotApplied { state: None }); }; if state != InstallState::Applied.as_str() { return Ok(InstalledRules::NotApplied { state: Some(state) }); } let version = version.and_then(|v| u32::try_from(v).ok()); - // A non-null `source` can only come from a joined row, which can only - // join on a non-null `rules_version` — so the version is present here. - // Both are still matched explicitly rather than assumed. - if source.as_deref() == Some(RULES_SOURCE_RLM) { - if let Some(version) = version { - return Ok(InstalledRules::RlmAuthored { version }); - } + let in_force = in_force_raw.and_then(|v| u32::try_from(v).ok()); + let Some(installed_version) = version else { + // An `applied` row with no recorded rule version cannot be admitted: + // there is no vector it can be shown to have landed. + return Ok(InstalledRules::NotRlmAuthored { + version: None, + provenance: source, + }); + }; + if source.as_deref() != Some(RULES_SOURCE_RLM) { + return Ok(InstalledRules::NotRlmAuthored { + version: Some(installed_version), + provenance: source, + }); + } + if in_force_source.as_deref() == Some(RULES_SOURCE_RLM) { + return Ok(InstalledRules::RlmAuthored { + version: installed_version, + }); } - Ok(InstalledRules::NotRlmAuthored { - version, - provenance: source, + // The install landed an RLM vector and something else is in force. A rule + // row for the install's version exists (the join matched), so the version + // in force exists too; the fallback is unreachable and only avoids a + // panic in a read that must stay total. + Ok(InstalledRules::SupersededByOperator { + installed: installed_version, + in_force: in_force.unwrap_or(installed_version), + provenance: in_force_source.unwrap_or_else(|| "absent".to_owned()), }) } diff --git a/crates/proof-topic-install/tests/install_engine.rs b/crates/proof-topic-install/tests/install_engine.rs index a7354b769..ceda9b9ad 100644 --- a/crates/proof-topic-install/tests/install_engine.rs +++ b/crates/proof-topic-install/tests/install_engine.rs @@ -435,6 +435,117 @@ async fn the_admission_read_binds_the_install_to_the_version_it_recorded() { tp.drop_schema().await.expect("drop"); } +/// An `operator` edit that supersedes the RLM's vector refuses the topic. +/// +/// The other direction of the same defect: the install recorded an +/// RLM-authored version, but the vector **in force** is an operator's. A gate +/// that bound only the install's version would admit the topic and serve rules +/// no RLM wrote. +#[tokio::test] +async fn an_operator_edit_in_force_refuses_the_topic() { + use proof_rlm::RuleSource; + use proof_task::ChecklistRule; + use proof_topic_install::{installed_rules, InstalledRules}; + + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let doc = topic("tb4"); + let installer = Installer { + pool: &pool, + store: &store, + }; + installer + .install( + &request(&doc, §ion("tb4")), + SetupSummary::NotDriven { reason: "x".into() }, + ) + .await + .expect("install"); + + // The RLM authors version 2, and an install records it. + let rlm = store + .current_rules("tb4") + .await + .expect("rules") + .expect("v1") + .next( + RuleSource::Rlm, + vec![ChecklistRule { + id: "rlm-1".into(), + text: "the RLM's rule".into(), + }], + ) + .expect("v2"); + store.put_rules(&rlm).await.expect("write v2"); + installer + .install( + &request(&doc, §ion("tb4")), + SetupSummary::Baselined { + rules_version: 2, + baseline_primary: "0.5".into(), + }, + ) + .await + .expect("re-install"); + assert_eq!( + installed_rules(&pool, "tb4").await.expect("read"), + InstalledRules::RlmAuthored { version: 2 }, + "the install landed the RLM's vector" + ); + + // An operator edit supersedes it. Both halves of the gate have to hold: + // the installed version is still `rlm`, but it is no longer in force. + let edited = store + .current_rules("tb4") + .await + .expect("rules") + .expect("v2") + .next( + RuleSource::Operator, + vec![ChecklistRule { + id: "hand-1".into(), + text: "an operator's rule".into(), + }], + ) + .expect("v3"); + store.put_rules(&edited).await.expect("write v3"); + assert_eq!( + installed_rules(&pool, "tb4").await.expect("read"), + InstalledRules::SupersededByOperator { + installed: 2, + in_force: 3, + provenance: "operator".into(), + }, + "an operator vector in force is not an admission, even when the install landed an RLM one" + ); + + // An RLM rewrite of its own rules stays admitted: that is the autonomy this + // gate protects, not something it may refuse. + let rewritten = store + .current_rules("tb4") + .await + .expect("rules") + .expect("v3") + .next( + RuleSource::Rlm, + vec![ChecklistRule { + id: "rlm-2".into(), + text: "the RLM rewrote its own rule".into(), + }], + ) + .expect("v4"); + store.put_rules(&rewritten).await.expect("write v4"); + assert_eq!( + installed_rules(&pool, "tb4").await.expect("read"), + InstalledRules::RlmAuthored { version: 2 }, + "an RLM rewrite (rlm -> rlm) stays admitted" + ); + + tp.drop_schema().await.expect("drop"); +} + /// A re-run resumes: migrations already in the journal are skipped, the rules /// version is not bumped, and the routes are not duplicated. #[tokio::test] From ec13bb4a0b61a4e7e0ea2ec6f209e5a1238483a0 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 18:40:50 +0000 Subject: [PATCH 04/42] fix(proof): the hardcode guard counts braces in code, not in text MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Greptile found the new guard's own blind spot: the stripper counted `{` and `}` in raw bytes, so a brace inside a `//` comment, a `"…"` literal, an `r#"…"#` block, or a macro's input left the depth non-zero after a `#[cfg(test)]` module closed. Every line after it was then dropped from the scan, and a prohibited topic literal placed there passed the guard. `mask_non_code` now blanks comments, string bodies (normal, byte, and raw), and character literals before any depth is computed, keeping byte offsets and newlines so line-by-line pairing with the original source holds. The attribute and `mod` markers are read from the masked line too, so a `mod ` inside a string is not mistaken for a module. A char literal is distinguished from a lifetime (`'a` is code and stays), and an unbalanced brace in text can no longer end a module early or keep one open. The regression test drives all four shapes Greptile named, each followed by production code carrying a literal, and asserts both that the literal is seen and that the module is still stripped. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- crates/proof-topic-bundle/src/lib.rs | 262 +++++++++++++++++++++++++-- 1 file changed, 247 insertions(+), 15 deletions(-) diff --git a/crates/proof-topic-bundle/src/lib.rs b/crates/proof-topic-bundle/src/lib.rs index 78ef5b5e0..8dd162db3 100644 --- a/crates/proof-topic-bundle/src/lib.rs +++ b/crates/proof-topic-bundle/src/lib.rs @@ -1315,12 +1315,177 @@ mod tests { .to_lowercase() } + /// Blank every byte that is **not** Rust code structure: comments, string + /// bodies (including raw and byte strings), and character literals. + /// + /// This is what makes brace counting honest. A `{` inside a `//` comment, + /// a `"…"` literal, or an `r#"…"#` block is *text*, not a delimiter; left + /// in place it would drive the depth counter and leave a `#[cfg(test)]` + /// module looking unclosed, so every line after it would be dropped from + /// the scan. That is precisely how a prohibited literal could hide. + /// + /// Masked bytes become spaces so byte offsets and line breaks survive + /// (a `//` comment ends at its newline, which is kept). + /// + /// Deliberately not a full Rust parser: it is a small lexer for the four + /// constructs that can contain a brace. It errs toward masking, which + /// makes the scan *more* inclusive of real code, never less. + /// + /// One function, not four: the cases are mutually exclusive branches of a + /// single left-to-right scan, and splitting them would mean re-deriving + /// "am I at a comment / a raw string / a string / a char?" in each helper. + #[allow(clippy::too_many_lines)] + fn mask_non_code(source: &str) -> String { + let bytes = source.as_bytes(); + let mut out = vec![b' '; bytes.len()]; + let mut i = 0usize; + while i < bytes.len() { + // Line comment: mask to end of line (newline kept). + if bytes[i] == b'/' && i + 1 < bytes.len() && bytes[i + 1] == b'/' { + while i < bytes.len() && bytes[i] != b'\n' { + i += 1; + } + continue; + } + // Block comment (nesting is legal in Rust). + if bytes[i] == b'/' && i + 1 < bytes.len() && bytes[i + 1] == b'*' { + let mut depth = 0usize; + while i < bytes.len() { + if bytes[i] == b'/' && i + 1 < bytes.len() && bytes[i + 1] == b'*' { + depth += 1; + i += 2; + continue; + } + if bytes[i] == b'*' && i + 1 < bytes.len() && bytes[i + 1] == b'/' { + depth -= 1; + i += 2; + if depth == 0 { + break; + } + continue; + } + if bytes[i] == b'\n' { + out[i] = b'\n'; + } + i += 1; + } + continue; + } + // Raw string: r"…", r#"…"#, br#"…"#, rb#"…"#. + let raw_at = { + let rest = &bytes[i..]; + let (skip, hashes) = if rest.starts_with(b"br") || rest.starts_with(b"rb") { + (2usize, 0usize) + } else if rest.starts_with(b"r") { + (1usize, 0usize) + } else { + (0usize, 0usize) + }; + if skip == 0 { + None + } else { + let mut h = hashes; + while i + skip + h < bytes.len() && bytes[i + skip + h] == b'#' { + h += 1; + } + if i + skip + h < bytes.len() && bytes[i + skip + h] == b'"' { + Some((skip + h + 1, h)) + } else { + None + } + } + }; + if let Some((body_start, hashes)) = raw_at { + i += body_start; + // Closing delimiter: `"` followed by `hashes` `#`. + loop { + if i >= bytes.len() { + break; + } + if bytes[i] == b'"' { + let mut k = i + 1; + let mut seen = 0usize; + while seen < hashes && k < bytes.len() && bytes[k] == b'#' { + seen += 1; + k += 1; + } + if seen == hashes { + i = k; + break; + } + } + if bytes[i] == b'\n' { + out[i] = b'\n'; + } + i += 1; + } + continue; + } + // Normal string or byte string. + let quote_at = if bytes[i] == b'"' { + Some(i) + } else if bytes[i] == b'b' && i + 1 < bytes.len() && bytes[i + 1] == b'"' { + Some(i + 1) + } else { + None + }; + if let Some(q) = quote_at { + i = q + 1; + while i < bytes.len() { + if bytes[i] == b'\\' { + i += 2; + continue; + } + if bytes[i] == b'"' { + i += 1; + break; + } + if bytes[i] == b'\n' { + out[i] = b'\n'; + } + i += 1; + } + continue; + } + // Character literal or a lifetime. A lifetime (`'a`) is code and + // must be kept; a char literal's contents are masked. The + // discriminator: a char literal closes within a few bytes, a + // lifetime is followed by an identifier. + if bytes[i] == b'\'' { + let closed = { + let mut j = i + 1; + if j < bytes.len() && bytes[j] == b'\\' { + j += 2; + while j < bytes.len() && bytes[j] != b'\'' { + j += 1; + } + j + 1 + } else if j + 1 < bytes.len() && bytes[j + 1] == b'\'' { + j + 2 + } else { + 0 + } + }; + if closed > 0 && closed <= bytes.len() { + i = closed; + continue; + } + } + out[i] = bytes[i]; + i += 1; + } + String::from_utf8_lossy(&out).into_owned() + } + /// Strip `#[cfg(test)] mod …` blocks from `source`, structurally. /// - /// Brace depth, not "everything after the first marker". A file may carry - /// a `#[cfg(test)]` attribute on a **method** (the guest's `Tail::bytes`) - /// with production code after it; splitting on the first marker would - /// declare that production code test-only and stop guarding it. + /// Brace depth over **code only** ([`mask_non_code`]), not "everything + /// after the first marker". A file may carry a `#[cfg(test)]` attribute on + /// a **method** (the guest's `Tail::bytes`) with production code after it; + /// splitting on the first marker would declare that production code + /// test-only and stop guarding it. And counting raw bytes would let a + /// brace in a comment or a string unbalance the depth, dropping every + /// later line from the scan. /// /// Only `mod` items are removed. A `#[cfg(test)]` on anything else is /// left in place deliberately: the scan then sees test-only code and @@ -1328,14 +1493,21 @@ mod tests { /// moving the fixture into a `mod tests`. Over-scanning is a false alarm; /// under-scanning is the hole this guards against. fn production_source(source: &str) -> String { + let masked = mask_non_code(source); + let masked_lines: Vec<&str> = masked.lines().collect(); let mut kept: Vec<&str> = Vec::new(); let mut depth_test: Option = None; let mut brace_depth = 0usize; let mut pending_cfg_test = false; - for line in source.lines() { - let trimmed = line.trim(); + for (idx, line) in source.lines().enumerate() { + // Braces are counted from the masked line, so a brace in a + // comment or a literal cannot move the depth. The attribute and + // `mod` markers are read from the masked line too, for the same + // reason (a `mod ` inside a string is not a module). + let code = masked_lines.get(idx).copied().unwrap_or(""); + let trimmed = code.trim(); if pending_cfg_test { - if trimmed.starts_with("mod ") && trimmed.contains('{') { + if trimmed.starts_with("mod ") && code.contains('{') { depth_test = Some(brace_depth); pending_cfg_test = false; } else if trimmed.ends_with(';') { @@ -1343,18 +1515,15 @@ mod tests { // file, which the caller's list already excludes. pending_cfg_test = false; continue; - } else if !trimmed.is_empty() - && !trimmed.starts_with("//") - && !trimmed.starts_with('#') - { + } else if !trimmed.is_empty() { // The attribute is on a non-`mod` item (a method, say): // keep it, so the scan still covers what follows. pending_cfg_test = false; } } - if trimmed == "#[cfg(test)]" || trimmed.starts_with("#[cfg(test)]") { + if trimmed.starts_with("#[cfg(test)]") { pending_cfg_test = true; - if trimmed.contains("mod ") && trimmed.contains('{') { + if trimmed.contains("mod ") && code.contains('{') { depth_test = Some(brace_depth); pending_cfg_test = false; } @@ -1363,8 +1532,8 @@ mod tests { if depth_test.is_none() { kept.push(line); } - brace_depth = brace_depth.saturating_add(line.matches('{').count()); - for _ in 0..line.matches('}').count() { + brace_depth = brace_depth.saturating_add(code.matches('{').count()); + for _ in 0..code.matches('}').count() { brace_depth = brace_depth.saturating_sub(1); if depth_test == Some(brace_depth) { depth_test = None; @@ -1485,6 +1654,69 @@ mod tests { ); } + /// A brace that is *text* cannot unbalance the strip and hide code after a + /// test module. + /// + /// The defect this pins: the stripper counted `{` / `}` in raw bytes, so a + /// brace inside a comment, a string, a raw string, or a macro's input left + /// the depth non-zero after the test module closed — and every line after + /// it was silently dropped from the scan. A prohibited literal placed + /// there passed the guard. + #[test] + fn braces_inside_text_do_not_hide_code_from_the_guard() { + // Each case: a `#[cfg(test)]` module whose body contains a brace that + // is *not* a delimiter, followed by production code carrying a + // literal. + let cases: [(&str, &str); 4] = [ + ( + "comment", + "#[cfg(test)]\nmod m {\n // an unbalanced brace in a comment: {\n}\nfn later() { let s = \"tbench\"; }\n", + ), + ( + "normal string", + "#[cfg(test)]\nmod m {\n fn f() { let s = \"a { brace\"; }\n}\nfn later() { let s = \"tbench\"; }\n", + ), + ( + "raw string", + "#[cfg(test)]\nmod m {\n fn f() { let s = r#\"a { brace\"#; }\n}\nfn later() { let s = \"tbench\"; }\n", + ), + ( + "macro input", + "#[cfg(test)]\nmod m {\n fn f() { println!(\"{{ literal brace\"); }\n}\nfn later() { let s = \"tbench\"; }\n", + ), + ]; + for (label, source) in cases { + let stripped = production_source(source); + assert!( + stripped.to_lowercase().contains("tbench"), + "a brace in a {label} must not hide the production code after the test module: \ + {stripped:?}" + ); + assert!( + !stripped.contains("mod m"), + "the test module is still stripped in the {label} case: {stripped:?}" + ); + } + } + + /// Masking removes exactly the non-code bytes and keeps structure. + #[test] + fn masking_blanks_text_and_keeps_code() { + let masked = mask_non_code("let a = 1; // { }\nlet b = \"}{ x\";\nlet c = 'x';\n"); + assert!( + !masked.contains("}{ x"), + "string bodies are masked: {masked:?}" + ); + assert!( + !masked.contains("// { }"), + "comment bodies are masked: {masked:?}" + ); + assert!(masked.contains("let a = 1;"), "{masked:?}"); + assert!(masked.contains("let c ="), "{masked:?}"); + // Newlines survive, so line-by-line pairing with the original holds. + assert_eq!(masked.lines().count(), 3, "{masked:?}"); + } + #[test] fn the_schema_key_lists_match_the_type() { let bundle = tb4(); From e36d340f0b5aa2091ccec00922be69384566303f Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 18:43:16 +0000 Subject: [PATCH 05/42] test(proof): run the structural strip over every product module MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The crate-local guards in `proof-rlm`, `proof-experiment`, and `proof-topic-install` strip test code with `split("#[cfg(test)]").next()`, which stops at the first marker even when it annotates a method rather than a module. Their sources are safe today — each marker starts a trailing `mod tests` — but the strip is the same shape that hid production code in the guest's `runner.rs`, so a literal added after such a marker would pass them. Running those sources through the structural strip here closes that latent gap without rewriting four crates' tests: the same files are now scanned by a strip that removes only `#[cfg(test)] mod` items and counts braces in code rather than in text. Proven non-vacuous by injecting a literal into `proof-rlm/src/lib.rs`, which the guard now names. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- crates/proof-topic-bundle/src/lib.rs | 59 +++++++++++++++++++++++++--- 1 file changed, 53 insertions(+), 6 deletions(-) diff --git a/crates/proof-topic-bundle/src/lib.rs b/crates/proof-topic-bundle/src/lib.rs index 8dd162db3..781f7c273 100644 --- a/crates/proof-topic-bundle/src/lib.rs +++ b/crates/proof-topic-bundle/src/lib.rs @@ -1551,12 +1551,22 @@ mod tests { /// Every product module that decides what a topic may do, and must /// therefore be **topic-agnostic**. /// - /// Listed by directory rather than one file at a time: a new module in one - /// of these crates is guarded the moment it is added, instead of being - /// silently unguarded until someone remembers this list. That is how - /// `proof-challenge/src/topic_routes.rs` came to be missing here — the - /// hand-maintained list, not the check, was the hole. - const PRODUCT_MODULES: [(&str, &str); 10] = [ + /// Listed one file at a time rather than by globbing the directories: + /// `include_str!` needs literal paths, and a glob would silently widen the + /// guard's surface when a new file lands. The cost is that a new product + /// module has to be added here — which is why + /// `the_product_branch_guard_catches_what_it_claims_to` asserts the list + /// still names the file the check was written for, and why the crates + /// carrying a module here are the ones whose whole surface is a product + /// branch. + /// + /// The modules the crates' own guards also cover are included: those + /// guards strip test code with `split("#[cfg(test)]").next()`, which + /// stops at the first marker even when it annotates a method rather than a + /// module. Running the same sources through the structural strip here + /// means a literal hidden after such a marker is caught even if the + /// crate-local guard misses it. + const PRODUCT_MODULES: [(&str, &str); 18] = [ ( "proof-challenge/src/topic_routes.rs", include_str!("../../proof-challenge/src/topic_routes.rs"), @@ -1597,10 +1607,47 @@ mod tests { "proof-rlm/src/runner.rs", include_str!("../../proof-rlm/src/runner.rs"), ), + ( + "proof-rlm/src/lib.rs", + include_str!("../../proof-rlm/src/lib.rs"), + ), + ( + "proof-rlm/src/gate.rs", + include_str!("../../proof-rlm/src/gate.rs"), + ), + ( + "proof-rlm/src/rules.rs", + include_str!("../../proof-rlm/src/rules.rs"), + ), + ( + "proof-rlm/src/state.rs", + include_str!("../../proof-rlm/src/state.rs"), + ), + ( + "proof-experiment/src/lib.rs", + include_str!("../../proof-experiment/src/lib.rs"), + ), + ( + "proof-experiment/src/policy.rs", + include_str!("../../proof-experiment/src/policy.rs"), + ), + ( + "proof-topic-install/src/lib.rs", + include_str!("../../proof-topic-install/src/lib.rs"), + ), + ( + "proof-topic-install/src/handler.rs", + include_str!("../../proof-topic-install/src/handler.rs"), + ), ]; /// The literals a product branch may not carry: a topic id, a benchmark /// name, or a results-contract id. + /// + /// `harbor-trials` is on the list even though `proof-results` legitimately + /// defines the contract id: that crate is not in [`PRODUCT_MODULES`] + /// because the id **is** its interface — a signed document pins it — while + /// no module here may branch on it. const FORBIDDEN_LITERALS: [&str; 5] = [ "tbench", "tb4", From 3ce29c8f4a40566badaa0e09cc40390adb369fcf Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 18:48:30 +0000 Subject: [PATCH 06/42] refactor(guest): the adaptor names no topic in its contract constant The Rust side already renamed `CONTRACT_TBENCH_HARBOR` to `CONTRACT_HARBOR_TRIALS_LEGACY`, keeping the *value* because it is a signed wire spelling while dropping the topic name from the identifier. The guest adaptor still carried `CONTRACT_TBENCH`, which names a topic the harness does not know. Renamed to `CONTRACT_HARBOR_TRIALS_LEGACY`, matching the Rust constant and its rationale. The string is untouched: a topic signed before the generic id existed pins `tbench-harbor-v1` in its signed `constraints.params.results_contract`, and a signed document cannot be edited. Nothing branches on a topic; the harness accepts both spellings. The deploy gate `assert-harbor-runner-results-emit.sh` asserts the new identifier, and the adaptor suites (148 Python tests, the shell suite) pass. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- .../rlm_fc_in_guest_harbor/harness/summarize.py | 15 ++++++++++----- .../tests/test_summarize.py | 2 +- .../scripts/assert-harbor-runner-results-emit.sh | 2 +- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/deploy/guest/runners/rlm_fc_in_guest_harbor/harness/summarize.py b/deploy/guest/runners/rlm_fc_in_guest_harbor/harness/summarize.py index 5717c36f8..98af7b69a 100755 --- a/deploy/guest/runners/rlm_fc_in_guest_harbor/harness/summarize.py +++ b/deploy/guest/runners/rlm_fc_in_guest_harbor/harness/summarize.py @@ -77,12 +77,17 @@ POLICY_ZERO = "zero" EXCEPTION_POLICIES = (POLICY_FAIL, POLICY_ZERO) CONTRACT_HARBOR_TRIALS = "harbor-trials-v1" -# Legacy alias for the same family, kept because it is a **wire value**: a -# topic signed before the generic id existed pins this in its +# Legacy wire value of CONTRACT_HARBOR_TRIALS, kept because it is a **wire +# value**: a topic signed before the generic id existed pins this in its # `constraints.params.results_contract`, and a signed document cannot be # edited. New topics pin `harbor-trials-v1`; nothing here branches on a topic. -CONTRACT_TBENCH = "tbench-harbor-v1" -HARBOR_CONTRACTS = (CONTRACT_HARBOR_TRIALS, CONTRACT_TBENCH) +# +# The constant name is deliberately topic-neutral, matching the Rust side +# (`proof_results::CONTRACT_HARBOR_TRIALS_LEGACY`): the value is a +# compatibility spelling, not a topic this harness knows. Do not rename the +# string — it is signed topic data. +CONTRACT_HARBOR_TRIALS_LEGACY = "tbench-harbor-v1" +HARBOR_CONTRACTS = (CONTRACT_HARBOR_TRIALS, CONTRACT_HARBOR_TRIALS_LEGACY) def _fail(msg: str, code: int = 2) -> None: @@ -124,7 +129,7 @@ def results_contract(pin: str) -> str: if name not in HARBOR_CONTRACTS: _fail( f"results_contract {name!r} is not a Harbor trial contract " - f"({CONTRACT_HARBOR_TRIALS} / {CONTRACT_TBENCH})" + f"({CONTRACT_HARBOR_TRIALS} / {CONTRACT_HARBOR_TRIALS_LEGACY})" ) return name diff --git a/deploy/guest/runners/rlm_fc_in_guest_harbor/tests/test_summarize.py b/deploy/guest/runners/rlm_fc_in_guest_harbor/tests/test_summarize.py index 3555dc19d..711bb459e 100644 --- a/deploy/guest/runners/rlm_fc_in_guest_harbor/tests/test_summarize.py +++ b/deploy/guest/runners/rlm_fc_in_guest_harbor/tests/test_summarize.py @@ -1368,7 +1368,7 @@ def test_invalid_utf8_and_escapes_use_encoded_json_size(self) -> None: trials, "", 0, "harbor", "", "", summarize.POLICY_FAIL ) inflated = summarize.build_results( - report, trials, "", summarize.CONTRACT_TBENCH + report, trials, "", summarize.CONTRACT_HARBOR_TRIALS_LEGACY ) self.assertGreater( summarize.results_payload_bytes(inflated, []), diff --git a/deploy/scripts/assert-harbor-runner-results-emit.sh b/deploy/scripts/assert-harbor-runner-results-emit.sh index b6d825f7a..4d1211486 100755 --- a/deploy/scripts/assert-harbor-runner-results-emit.sh +++ b/deploy/scripts/assert-harbor-runner-results-emit.sh @@ -32,7 +32,7 @@ need "$SUMMARIZE" 'def write_results_next_to_report' # The generic contract id is what an un-pinned topic gets; the tbench-named id # is a legacy wire value (a signed topic may still pin it) and stays accepted. need "$SUMMARIZE" 'CONTRACT_HARBOR_TRIALS = "harbor-trials-v1"' -need "$SUMMARIZE" 'CONTRACT_TBENCH = "tbench-harbor-v1"' +need "$SUMMARIZE" 'CONTRACT_HARBOR_TRIALS_LEGACY = "tbench-harbor-v1"' need "$SUMMARIZE" 'name = (pin or "").strip() or CONTRACT_HARBOR_TRIALS' need "$SUMMARIZE" 'results.json first' need "$SUMMARIZE" 'write_results_next_to_report(out, report, trials, log_tail, secrets)' From bf40ca17a359559e0c87e0ba577cd5d4fb665ea9 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 18:49:50 +0000 Subject: [PATCH 07/42] test(proof): guard every product module in the challenge stack MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The guard covered ten modules chosen by hand. Sixteen more decide what a topic may do and were unguarded: the install executor's engine, routes, section reader, and gate; the guest's fetch and staging; the VM agent's router, auth, hypervisor, and stamp; the gateway's auth, attestation, and proxy paths; and `gateway-core/src/lib.rs`. All thirty-two are now scanned with the structural strip, and the tree is clean for every one of them today — this closes the surface a future edit could put a literal back into. The guard's self-test still asserts it names the file the check was written for and that an injected literal is caught. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- crates/proof-topic-bundle/src/lib.rs | 58 +++++++++++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/crates/proof-topic-bundle/src/lib.rs b/crates/proof-topic-bundle/src/lib.rs index 781f7c273..852dacb0a 100644 --- a/crates/proof-topic-bundle/src/lib.rs +++ b/crates/proof-topic-bundle/src/lib.rs @@ -1566,7 +1566,7 @@ mod tests { /// module. Running the same sources through the structural strip here /// means a literal hidden after such a marker is caught even if the /// crate-local guard misses it. - const PRODUCT_MODULES: [(&str, &str); 18] = [ + const PRODUCT_MODULES: [(&str, &str); 32] = [ ( "proof-challenge/src/topic_routes.rs", include_str!("../../proof-challenge/src/topic_routes.rs"), @@ -1639,6 +1639,62 @@ mod tests { "proof-topic-install/src/handler.rs", include_str!("../../proof-topic-install/src/handler.rs"), ), + ( + "proof-topic-install/src/install.rs", + include_str!("../../proof-topic-install/src/install.rs"), + ), + ( + "proof-topic-install/src/routes.rs", + include_str!("../../proof-topic-install/src/routes.rs"), + ), + ( + "proof-topic-install/src/section.rs", + include_str!("../../proof-topic-install/src/section.rs"), + ), + ( + "proof-topic-install/src/gate.rs", + include_str!("../../proof-topic-install/src/gate.rs"), + ), + ( + "proof-vm-guest/src/fetch.rs", + include_str!("../../proof-vm-guest/src/fetch.rs"), + ), + ( + "proof-vm-guest/src/staging.rs", + include_str!("../../proof-vm-guest/src/staging.rs"), + ), + ( + "proof-vm-agent/src/lib.rs", + include_str!("../../proof-vm-agent/src/lib.rs"), + ), + ( + "proof-vm-agent/src/auth.rs", + include_str!("../../proof-vm-agent/src/auth.rs"), + ), + ( + "proof-vm-agent/src/hypervisor.rs", + include_str!("../../proof-vm-agent/src/hypervisor.rs"), + ), + ( + "proof-vm-agent/src/stamp.rs", + include_str!("../../proof-vm-agent/src/stamp.rs"), + ), + ( + "gateway-core/src/lib.rs", + include_str!("../../gateway-core/src/lib.rs"), + ), + ( + "gateway-core/src/admin_auth.rs", + include_str!("../../gateway-core/src/admin_auth.rs"), + ), + ( + "gateway-core/src/admin_attest.rs", + include_str!("../../gateway-core/src/admin_attest.rs"), + ), + ( + "gateway-core/src/proxy_paths.rs", + include_str!("../../gateway-core/src/proxy_paths.rs"), + ), ]; /// The literals a product branch may not carry: a topic id, a benchmark From 9b55fe732bffb149bd04b8e5afa6e3085dec71d9 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 18:56:18 +0000 Subject: [PATCH 08/42] docs: correct two stale claims about the RLM authorship path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both docs described the state before this stack shipped the install path, and both were wrong in the same direction: they understated what is enforced. `deploy/guest/runners/README.md` said a runner without a `propose_rules` entrypoint makes the agent propose the signed topic's own `checklist`. That echo was removed: it is exactly how an operator-cloned document gets recorded as `source = rlm`. The guest now refuses the job (Failed → 503, no row, nothing scored), and the row says so. `docs/COMPLETENESS.md` called topic installs a **skeleton** with the real install unimplemented (exit 3) and `enable` / `disable` / `seal` as stubs. All three shipped: `topic install` applies the bundle's RLM section and journals it, `seal` drives `mark_sealed`, and the gate rows live in `0026_proof_topic_gate.sql`. The row now records the three migrations, the mounted dynamic route table, the install-bound provenance gate (both the recorded version and the version in force), and — explicitly — that the six live gates are staged but **not** claimed green. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- deploy/guest/runners/README.md | 2 +- docs/COMPLETENESS.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/deploy/guest/runners/README.md b/deploy/guest/runners/README.md index 77e86d1fc..a7a40aea2 100644 --- a/deploy/guest/runners/README.md +++ b/deploy/guest/runners/README.md @@ -28,7 +28,7 @@ operator's view of it. |------|-----|--------------------------------------| | `run` (required) | `Baseline`, `Evaluate` | `report.json` — `{"primary_value": , "claim_holds": bool, "flops_used": , "evidence": {...}}`. **Evaluate** also writes the topic-defined complete results JSON (default `results.json`; pin `results_path` / `results_contract` in `constraints.params`). Missing or non-conforming on evaluate is fail-closed (no Done) | | `inspect` | `Inspect` (anti-cheat rules, **before any paid inference**) | `checklist.json` — `[{"id": "", "pass": bool, "evidence": "..."}]`; a rule left out is recorded **red** | -| `propose_rules` (optional) | `ProposeRules` | `rules.json` — `[{"id": "", "text": "..."}]`; without this entrypoint the agent proposes the signed topic's own `checklist` | +| `propose_rules` | `ProposeRules` (RLM authorship) | `rules.json` — `[{"id": "", "text": "..."}]`. **A runner whose topic must open needs this entrypoint**: without it the guest refuses the job (`Failed` → 503, no row, nothing scored), because there is **no** fallback that echoes the signed `checklist` back. Echoing it would let the control plane record the operator's own vector as `source = rlm`, which is an operator-cloned document masquerading as RLM authorship. The signed `checklist` stays the topic's version 1 with honest `topic_document` provenance, and only a run of this entrypoint advances the store to `rlm` — which the publish gate requires before a topic may be `open`. | A non-zero exit with no document, a missing document, a non-finite `primary_value`, a missing or non-conforming Evaluate `results.json`, or a diff --git a/docs/COMPLETENESS.md b/docs/COMPLETENESS.md index 2f1f752c4..d01b7f200 100644 --- a/docs/COMPLETENESS.md +++ b/docs/COMPLETENESS.md @@ -81,7 +81,7 @@ specs (`DESIGN_CHALLENGE.md`, `PRISM.md`) remain for `xtask` gates. Leftover | Inference offer | **v0** | Digest-pinned RLM **judge** backend (`proof-eval` / harvest call it). Pin `[inference]` defaults plus schema v1 / ceilings / modes / commitment. `config_commitment` hashes config knobs **and** `provider.base_url`; a topic that spoofs origin is **503** before lattice. Topic `require_judge_offer_commitment` is optional and not a miner bind. Live `InferenceOffer` is operator state. Auth is `PROOF_INFERENCE_API_KEY_FILE` staged as harvest `teacher.env` (never git, never `/v1/status`). Missing/closed/judge down / missing key → `can_score=false` / 503. No baked Qwen; architecture ≠ HF stays retired. | | Eval executor offer | **v0** | `crates/proof-executor`: live `1x` `EvalExecutorOffer` (Lium template, `machine_shape`, `max_proof_deadline_s`, digest, `config_commitment`, status) — a sibling of the judge offer, not the same document. Pin ceilings `eval_executor_schema_version` / `gpu_class = "1x"` / `max_proof_deadline_s_ceiling = 14400` / optional `allowed_lium_template_prefixes` / `eval_executor_commitment_alg`. Public on `GET /v1/status` + `GET /v1/proof/executor`; rotated via `POST /v1/admin/proof/executor` (in-memory until restart; boot from `PROOF_EVAL_EXECUTOR_OFFER_FILE`). Topic tighten-only `eval_executor.{require_offer_commitment, max_proof_deadline_s}`, no per-topic `machine_id`. Lium path: missing/closed/shape ≠ `1x` → `can_score=false` / 503; harvest rents the offer's digest-scoped template (raw Lium UUIDs refused under any allowlist; the resolver binds the template to `eval_image@digest`) at exactly `1x` (`rent_gpu_count ≠ 1` aborts pre-rent) and holds the run to the deadline (the deadline is the pod `timeout`, never clamped by the host fallback; harvest wait = deadline + grace; wrapper-cut run → 503 + `stdout_tail`, external SIGKILL named separately). `PROOF_HARVEST_TEMPLATE_ID` / `_GPU_COUNT` / `_DEADLINE_SECS` hot-swap under the pin ceilings; refused when the topic pins the offer commitment; the run request and row stamp the commitment of what actually ran. Sim does not consult it. No live Lium rent in CI. | | Topics | **done** | sr25519 under the `proof` trust-root key (`base-proof-topic-v1`). Admin `POST /v1/admin/proof/topics`. A topic must be sealed to `open`. | -| Topic installs (dynamic topics P0) | **skeleton** | `bins/proof-admin` wraps the **existing** publish path rather than adding a registry: `topic validate` runs the same acceptance `POST /v1/admin/proof/topics` runs (`TopicDocument::validate` + `verify_signature`, against `config/proof-pin.toml`), `topic install --dry-run` prints that publish call plus the host env, and `topic list` / `topic show` are a read-only view of the existing `proof_topic_version` rows (`RlmStore::latest_topics`, migration `0020`). Bundle schema v1 (`crates/proof-topic-bundle`) carries the signed document verbatim plus a `host` block that must **agree** with it (a contradiction is a reject); runner/pack/custom-id bindings are the document's own `constraints.params`. **Schema: `0024_proof_topic_alias.sql` only** — it adds `proof_topic_alias` plus a `BEFORE INSERT`/`UPDATE` trigger pair that fails closed when an alias would shadow a published slug (**publish-path integrity, not scoring math**); it does not `ALTER` or `DROP` anything, and `0020` tables keep their columns, keys, and grants. **No route change, no scoring change** — a real install is not implemented (exit 3), and `topic enable` / `disable` / `seal` are stubs (a topic's lifecycle is the document's `status`). No route change (P1), allocator change (P2), full install (P3), or removal of the compiled-in topic bindings (P4). **No compiled-in topic:** a topic's slug, alias, and custom id are its own signed data, and which topics exist is a fact about `proof_topic_version` + `proof_topic_install` — never a system default. The topic's behavior must be **RLM-authored** (`proof_rule_version.source = 'rlm'`, written only by the topic's own `propose_rules` job in its VM; the guest refuses to echo the signed checklist back) before an `open` document can be published or admitted at boot; the publish gate and the challenge's topic loader both read that provenance and fail closed. Shared challenge DB with a `topic_id` discriminant, and **metal `--env metal` is Owner-only behind `--owner-metal-ack` with staging first** (staging is never gated). **Topics are RLM-owned:** the bundle's `rlm` section (rules / migrations / apis / submission_format / scoring) is handed to the RLM verbatim and Rust never interprets it; guard tests fail the build if a topic id or a topic-specific rule/metric/format appears in the bundle crate's, the CLI's, or the challenge / gateway / orchestrator / guest product branches. | +| Topic installs (dynamic topics P0) | **partial** | `bins/proof-admin` drives the install: `topic validate` runs the same acceptance `POST /v1/admin/proof/topics` runs (`TopicDocument::validate` + `verify_signature`, against `config/proof-pin.toml`), `topic install` (real, not a stub) applies the bundle's RLM section — migrations under the deny-list, `apis` as topic-scoped routes, `rules` through the store — journals the result, and publishes last; `--drive-rlm --owner-approved` additionally provisions the topic VM and runs the RLM's `propose_rules` → paid baseline (no `--skip-baseline` for a real measurement); `topic install-log` reads the journal back. `topic enable` / `disable` / `seal` are **implemented** (gate rows in `0026_proof_topic_gate.sql`, seal in `crates/proof-topic-ops`), not stubs. Bundle schema v1 (`crates/proof-topic-bundle`) carries the signed document verbatim plus a `host` block that must **agree** with it (a contradiction is a reject); runner/pack/custom-id bindings are the document's own `constraints.params`. **Schema: `0024_proof_topic_alias.sql` (alias), `0025_proof_topic_install.sql` (install journal + `proof_topic_api` dynamic routes), `0026_proof_topic_gate.sql` (the operator disable switch)** — all append-only for `base_app`; none `ALTER`s or `DROP`s anything, so `0020` tables keep their columns, keys, and grants. The challenge mounts the topic route table (`challenge_router(state, topic_routes)`), so a topic's own registered routes are served from the DB rather than a compiled list. **No compiled-in topic:** a topic's slug, alias, and custom id are its own signed data, and which topics exist is a fact about `proof_topic_version` + `proof_topic_install` — never a system default. The topic's behavior must be **RLM-authored** (`proof_rule_version.source = 'rlm'`) before an `open` document can be published or admitted at boot: the gate reads the rule version the newest `applied` install **recorded** *and* the version in force, so a document-sourced vector or a later `operator` edit refuses, while the RLM rewriting its own rules stays admitted. The guest refuses to echo the signed checklist back, so a runner shipping no `propose_rules` entrypoint is `Failed` (503, no row) rather than silently scored under the operator's vector. Shared challenge DB with a `topic_id` discriminant, and **metal `--env metal` is Owner-only behind `--owner-metal-ack` with staging first** (staging is never gated). **Topics are RLM-owned:** the bundle's `rlm` section (rules / migrations / apis / submission_format / scoring) is handed to the RLM verbatim and Rust never interprets it; guard tests fail the build if a topic id or a topic-specific rule/metric/format appears in any of the 32 guarded product modules across the bundle crate, the CLI, and the challenge / gateway / orchestrator / guest. **Not yet proven live:** gates 1–6 (measured baseline → seal/`can_score` → miner `pf_`/Harbor/`results.json` → two concurrent submits ⇒ two VMs → topic disable fail-closed → gateway-Bearer publish with no rewrite proxy) are staged in the OWNER_LIVE checklist on PR #301 and are **not** claimed green. | | Holdout | **done** | Per-topic operator file (`PROOF_HOLDOUT_FILE`). Commitment in the topic document, never in the pin. `xtask proof-holdout --topic-id`. | | Live harvest | **partial** | `crates/proof-harvest` over `harvest-pod` stages `request.json`, `teacher.env`, `PROOF_PROXY_MODEL_DIR`, and `PROOF_HOLDOUT_STORE`. `PROOF_FORCE_SIM` is local-only. Live rent still needs a republished proof-eval digest (current pin still has the invalid HF default) plus operator-staged proxy dir + holdout shards. | | Configured allocation | **8000 bps** | Proof-weighted 20%/80% regardless of digest. Payout splits equally across currently `open` topics, then `wta` or `discovery`. Empty digest / missing evaluation prerequisites still fail closed. | From 724ff5b9c621cdc18425f4f7169c8c0a99147585 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 19:12:20 +0000 Subject: [PATCH 09/42] fix(proof): a hyphenated topic id can name a table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every real topic id is a hyphen slug (`[a-z0-9][a-z0-9-]{1,62}`), and a bare SQL identifier cannot contain a hyphen. The migration guard required a literal `{topic_id}_` prefix, so its requirement was **unsatisfiable**: `CREATE TABLE fixture-topic-v0_scratch` is a syntax error at the first `-`, and the underscore spelling was refused as unscoped. An operator running `topic install --drive-rlm` would provision the VM, run the paid baseline, and only then hit the deny-list — a paid run that could never publish. `topic_sql_prefix` maps `-` to `_`, and `is_topic_scoped` accepts that identifier-safe spelling alongside the literal one. It stays a boundary: ids contain no underscores, so the mapping is injective and two ids cannot collide on one prefix. A sibling topic's table, a `topic_*` name, and a `proof_*` object are all still refused. The refusal message also claimed `topic_*` was allowed, which the code has always refused; it now names the prefix that actually works. This was invisible in CI: the guard's own suite only used `tb4` (no hyphen), and the tests that would have caught it are DB-gated and CI has no Postgres. Found by running the DB-gated suites against a real database. Two `proof-admin` tests that had never executed now pass; one of them also asserted the topic's alias where the code correctly prints the custom id. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- bins/proof-admin/tests/cli.rs | 9 ++- crates/proof-topic-install/src/lib.rs | 4 +- crates/proof-topic-sql-guard/src/lib.rs | 46 +++++++++-- .../proof-topic-sql-guard/tests/sql_guard.rs | 78 ++++++++++++++++++- 4 files changed, 125 insertions(+), 12 deletions(-) diff --git a/bins/proof-admin/tests/cli.rs b/bins/proof-admin/tests/cli.rs index 34ee69a5e..08e6b5121 100644 --- a/bins/proof-admin/tests/cli.rs +++ b/bins/proof-admin/tests/cli.rs @@ -1193,7 +1193,14 @@ async fn the_registry_view_lists_what_the_scoring_path_persisted() { let listed: serde_json::Value = serde_json::from_str(&stdout(&out)).expect("json"); assert_eq!(listed[0]["topic_id"], "fixture-topic-v0"); assert_eq!(listed[0]["version"], 1); - assert_eq!(listed[0]["custom_id"], "fixture-alias"); + // `custom_id` is the runner-registry key the document names + // (`metric.custom_id`), not the topic's alias — the two are different + // mappings, and the alias is `fixture-alias`. + assert_eq!(listed[0]["custom_id"], "fixture_metric_v0"); + assert_ne!( + listed[0]["custom_id"], "fixture-alias", + "an alias is a slug lookup, not the custom id" + ); let out = run_db(&["--json", "topic", "show", "fixture-topic-v0"]); assert_eq!(code(&out), 0, "stderr={}", stderr(&out)); diff --git a/crates/proof-topic-install/src/lib.rs b/crates/proof-topic-install/src/lib.rs index 4c5742b20..22f19194f 100644 --- a/crates/proof-topic-install/src/lib.rs +++ b/crates/proof-topic-install/src/lib.rs @@ -67,8 +67,8 @@ pub use install::{ }; pub use proof_topic_sql_guard::{ blank_statements, check_migration, check_statement, is_topic_scoped, split_statements, - MigrationDenied, Statement, DENIED_DROP_KINDS, DENIED_FUNCTIONS, DENIED_OBJECTS, DENIED_VERBS, - OWNED_TABLES, OWNED_TABLE_PREFIX, + topic_sql_prefix, MigrationDenied, Statement, DENIED_DROP_KINDS, DENIED_FUNCTIONS, + DENIED_OBJECTS, DENIED_VERBS, OWNED_TABLES, OWNED_TABLE_PREFIX, }; pub use routes::{is_topic_id, PgTopicRoutes, Resolved, TopicRouteMux, TopicRouteSource}; pub use section::{ diff --git a/crates/proof-topic-sql-guard/src/lib.rs b/crates/proof-topic-sql-guard/src/lib.rs index cfceabf0f..1b485295e 100644 --- a/crates/proof-topic-sql-guard/src/lib.rs +++ b/crates/proof-topic-sql-guard/src/lib.rs @@ -18,9 +18,11 @@ //! `VACUUM` / `CLUSTER` / `REINDEX`, `SECURITY DEFINER` functions, and //! server-side file access (`pg_read_file`, `lo_import`, …). //! - **Namespace**: every table a statement creates, writes, or reads must be -//! inside the topic's own namespace (`{topic_id}_*`, `topic_*`, or -//! `{topic_id}.…`). Without this a topic could claim a generic name and -//! collide with the next topic's install, or read a sibling topic's rows. +//! inside the topic's own namespace (`{topic_sql_prefix}_*`, or +//! `{topic_sql_prefix}.…` — the id with `-` mapped to `_`, since a topic id +//! is a hyphen slug and a bare SQL identifier cannot contain a hyphen). +//! Without this a topic could claim a generic name and collide with the next +//! topic's install, or read a sibling topic's rows. //! //! # What this is not //! @@ -1043,6 +1045,25 @@ fn copy_chars( idx } +/// The SQL-safe spelling of a topic id. +/// +/// A topic id is a hyphen slug (`[a-z0-9][a-z0-9-]{1,62}`) and **may not +/// contain an underscore**, while a bare SQL identifier is +/// `[a-z_][a-z0-9_$]*` and **may not contain a hyphen** — the two alphabets do +/// not intersect except on `[a-z0-9]`. So the literal id can never prefix a +/// bare identifier: `CREATE TABLE fixture-topic-v0_scratch` is a syntax error +/// at the first `-`, and the guard's own `{topic_id}_*` requirement would be +/// unsatisfiable for every real topic. +/// +/// Mapping `-` → `_` gives the topic an identifier-safe prefix. It stays a +/// **boundary**: ids contain no underscores, so the mapping is injective — +/// two different ids cannot collide on one prefix, and a name scoped to +/// another topic's prefix cannot match this one. +#[must_use] +pub fn topic_sql_prefix(topic_id: &str) -> String { + topic_id.trim().to_ascii_lowercase().replace('-', "_") +} + /// Whether `name` is inside `topic_id`'s namespace. /// /// Two spellings are the topic's, and only two: @@ -1050,6 +1071,12 @@ fn copy_chars( /// - a `{topic_id}`-qualified name (`tb4.scores`, `tb4.runs`), or /// - a bare `{topic_id}_`-prefixed name (`tb4_scores`). /// +/// For an id containing a hyphen the identifier-safe form +/// ([`topic_sql_prefix`], `fixture-topic-v0` → `fixture_topic_v0`) is accepted +/// in both positions, because the literal id cannot appear in a bare SQL +/// identifier at all. The literal spelling stays accepted too, for the ids +/// that need no mapping and for quoted schema-qualified names. +/// /// # Why there is no generic `topic_` allowance /// /// An earlier revision also accepted any `topic_*` name, on the theory that a @@ -1072,14 +1099,15 @@ pub fn is_topic_scoped(name: &str, topic_id: &str) -> bool { return false; } let topic = topic_id.trim().to_ascii_lowercase(); + let sql_topic = topic_sql_prefix(&topic); let (schema, bare) = match n.split_once('.') { Some((s, b)) => (Some(s), b), None => (None, n.as_str()), }; - if schema == Some(topic.as_str()) { + if schema == Some(topic.as_str()) || schema == Some(sql_topic.as_str()) { return true; } - bare.starts_with(&format!("{topic}_")) + bare.starts_with(&format!("{topic}_")) || bare.starts_with(&format!("{sql_topic}_")) } /// Check a statement's text against every deny rule. @@ -1164,9 +1192,11 @@ fn check_text( return Err(deny( &name, &format!( - "a topic migration may only touch objects named {topic_id}_*, topic_*, or \ - {topic_id}.*; an unscoped name would collide with — or read — another \ - topic's install" + "a topic migration may only touch objects named {sql}_* (or {id}_*), \ + {sql}.* (or {id}.*); an unscoped name would collide with — or read — \ + another topic's install", + sql = topic_sql_prefix(topic_id), + id = topic_id ), )); } diff --git a/crates/proof-topic-sql-guard/tests/sql_guard.rs b/crates/proof-topic-sql-guard/tests/sql_guard.rs index 0cc80732b..84c414d31 100644 --- a/crates/proof-topic-sql-guard/tests/sql_guard.rs +++ b/crates/proof-topic-sql-guard/tests/sql_guard.rs @@ -14,7 +14,8 @@ use proof_topic_sql_guard::MigrationDenied; use proof_topic_sql_guard::{ - blank_statements, check_migration, is_topic_scoped, split_statements, OWNED_TABLES, + blank_statements, check_migration, is_topic_scoped, split_statements, topic_sql_prefix, + OWNED_TABLES, }; const TOPIC: &str = "tb4"; @@ -268,6 +269,81 @@ fn a_topics_own_namespace_is_allowed() { assert!(!is_topic_scoped("", "tb4")); } +/// A real topic id is a **hyphen slug**, and a bare SQL identifier cannot +/// contain a hyphen — so the guard has to accept the identifier-safe spelling +/// or no real topic could ever install a migration. +/// +/// The defect this pins: the guard required a literal `{topic_id}_` prefix. +/// Every live topic id is `[a-z0-9][a-z0-9-]{1,62}`, so the requirement was +/// unsatisfiable: `CREATE TABLE fixture-topic-v0_scratch` is a syntax error at +/// the first `-`, and the underscore spelling was refused. `--drive-rlm` would +/// provision the VM, run the paid baseline, and only then fail the install on +/// the deny-list — a paid run that could never publish. +#[test] +fn a_hyphenated_topic_id_has_an_identifier_safe_namespace() { + // The mapping is `-` → `_`. It is injective **over legal ids**: an id + // cannot contain an underscore (`[a-z0-9][a-z0-9-]{1,62}`), so two + // different real ids cannot collide on one prefix. + assert_eq!(topic_sql_prefix("fixture-topic-v0"), "fixture_topic_v0"); + assert_eq!(topic_sql_prefix("tb4"), "tb4"); + assert_ne!( + topic_sql_prefix("a-b"), + topic_sql_prefix("a-b-c"), + "different ids map to different prefixes" + ); + + // The identifier-safe spelling is inside the topic's namespace… + assert!(is_topic_scoped( + "fixture_topic_v0_scratch", + "fixture-topic-v0" + )); + assert!(is_topic_scoped("fixture_topic_v0.runs", "fixture-topic-v0")); + // …the literal spelling stays accepted where it is legal (quoted / schema) + assert!(is_topic_scoped("fixture-topic-v0.runs", "fixture-topic-v0")); + // …and a sibling is still refused, in both spellings. + assert!(!is_topic_scoped( + "fixture_topic_v1_scratch", + "fixture-topic-v0" + )); + assert!(!is_topic_scoped("other_topic_scratch", "fixture-topic-v0")); + assert!(!is_topic_scoped("topic_scratch", "fixture-topic-v0")); +} + +/// The whole pipeline, end to end: a hyphenated topic's migration is +/// **allowed**, and a sibling's table is still refused. +#[test] +fn a_hyphenated_topics_migration_is_admitted() { + let topic = "fixture-topic-v0"; + // The exact statement the operator fixture carries. + allowed_for("CREATE TABLE fixture_topic_v0_scratch (id TEXT)", topic); + allowed_for( + "CREATE INDEX fixture_topic_v0_scratch_id ON fixture_topic_v0_scratch (id)", + topic, + ); + allowed_for( + "INSERT INTO fixture_topic_v0_scratch (id) VALUES ('a')", + topic, + ); + + // A sibling topic's table is not this topic's, however similar. + for sql in [ + "CREATE TABLE fixture_topic_v1_scratch (id TEXT)", + "CREATE TABLE topic_scratch (id TEXT)", + "SELECT id FROM proof_rule_version", + ] { + assert!( + check_migration(sql, topic).is_err(), + "{sql:?} must be refused for {topic}" + ); + } +} + +/// [`allowed`], for an arbitrary topic id. +fn allowed_for(sql: &str, topic: &str) { + check_migration(sql, topic) + .unwrap_or_else(|e| panic!("{sql:?} must be allowed for {topic}: {e}")); +} + /// A quoted identifier still names an object, so quoting cannot smuggle a /// denied table past the scanner. #[test] From 4977db81133768c66881fe25cb9b1ad4c358d48e Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 19:21:47 +0000 Subject: [PATCH 10/42] test(proof): a hyphenated topic id installs its migrations end to end MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The install-engine suite exercised only `tb4`, which is why the guard's unsatisfiable prefix requirement (fixed in 724ff5b9) went unnoticed: `tb4` is both a legal id and a legal bare SQL identifier, so `tb4_scratch` happened to satisfy it. Every real id is a hyphen slug, where it cannot. This test drives the whole path for `fixture-topic-v0`: the migration applies, its table exists in the schema, and a sibling's table, a `topic_*` name, and a `proof_*` object are all still refused — the fix widened the namespace rather than removing it. Proven non-vacuous: with the guard's pre-fix logic restored, the test fails with exactly the LIVE blocker's refusal. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- .../tests/install_engine.rs | 78 +++++++++++++++++++ 1 file changed, 78 insertions(+) diff --git a/crates/proof-topic-install/tests/install_engine.rs b/crates/proof-topic-install/tests/install_engine.rs index ceda9b9ad..7cf066e1c 100644 --- a/crates/proof-topic-install/tests/install_engine.rs +++ b/crates/proof-topic-install/tests/install_engine.rs @@ -114,6 +114,84 @@ fn section(id: &str) -> String { .replace("{id}", id) } +/// A **hyphenated** topic id installs its migrations end to end. +/// +/// Every real topic id is a hyphen slug (`[a-z0-9][a-z0-9-]{1,62}`), and a +/// bare SQL identifier cannot contain a hyphen. The defect this pins: the +/// migration guard required a literal `{topic_id}_` prefix, so its requirement +/// was unsatisfiable — `CREATE TABLE real-topic_scratch` is a syntax error at +/// the first `-`, and the underscore spelling was refused as unscoped. No real +/// topic could install a migration, and `topic install --drive-rlm` would only +/// discover it *after* the paid baseline. +/// +/// The suite's other tests all use `tb4`, which is why this went unnoticed. +#[tokio::test] +async fn a_hyphenated_topic_id_installs_its_migrations() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let id = "fixture-topic-v0"; + let doc = topic(id); + let installer = Installer { + pool: &pool, + store: &store, + }; + // The table the migration creates, spelled with the identifier-safe + // prefix the guard maps the id to. `section()` appends `_scratch`. + let table = "fixture_topic_v0_scratch"; + let report = installer + .install( + &request(&doc, §ion("fixture_topic_v0")), + SetupSummary::Skipped { + reason: "--skip-baseline".into(), + }, + ) + .await + .expect("a hyphenated topic's bundle installs"); + + assert_eq!(report.topic_id, id); + assert_eq!( + report.migrations_applied, + ["0001_scratch", "0002_index"], + "both migrations apply for a hyphenated id" + ); + // The migration really ran: its table exists in this test's schema. + let exists: Option = + sqlx::query_scalar(&format!("SELECT to_regclass('{table}')::text")) + .fetch_one(&pool) + .await + .expect("probe the table"); + assert_eq!( + exists.as_deref(), + Some(table), + "the hyphenated topic's migration created its table" + ); + + // And a migration reaching a sibling topic is still refused, in both + // spellings — the fix widened the namespace, it did not remove it. + for sql in [ + "CREATE TABLE fixture_topic_v1_scratch (id TEXT)", + "CREATE TABLE topic_scratch (id TEXT)", + "CREATE TABLE proof_rule_version (id TEXT)", + ] { + let section = format!(r#"{{"migrations": [{{"name": "0003_bad", "sql": "{sql}"}}]}}"#); + let err = installer + .install( + &request(&doc, §ion), + SetupSummary::Skipped { reason: "x".into() }, + ) + .await + .expect_err("a sibling's table is not this topic's"); + assert!( + matches!(err, proof_topic_install::InstallError::MigrationDenied(_)), + "{sql:?} must be refused by the guard, got {err:?}" + ); + } + + tp.drop_schema().await.expect("drop"); +} + /// The happy path: every step applies, and the journal records it. #[tokio::test] async fn a_permitted_bundle_installs_and_the_journal_records_it() { From 6b891872f73690e6839c6e598e33fb2969018627 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 19:30:33 +0000 Subject: [PATCH 11/42] fix(proof): a quoted hyphenated identifier is one name to the guard MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Greptile found that the literal spelling I documented in 724ff5b9 is unreachable. `is_topic_scoped` accepts `"fixture-topic-v0_scratch"`, but `tokens` split on every character that is not alphanumeric, `_`, or `.`, so the `-` broke the quoted run into `fixture` / `topic` / `v0_scratch` — none of which is inside the topic's namespace, so a legal quoted name was refused as unscoped. `tokens` now keeps a double-quoted run whole, stripping the quotes (and folding `""` to one literal quote) so the token is the name the database stores and the deny rules match it exactly as before. The SQL-safe underscore spelling always worked; this makes the documented literal one work too. Non-vacuous: with the tokenizer reverted, both new tests fail — the quoted-name test on the refusal, and the escaped-quote test showing the name tokenized as `["a"]`. Quoting remains no escape hatch: a sibling, a `topic_*` name, and a `proof_*` object are all still refused when quoted. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- crates/proof-topic-sql-guard/src/lib.rs | 31 ++++++++++++- .../proof-topic-sql-guard/tests/sql_guard.rs | 46 ++++++++++++++++++- 2 files changed, 74 insertions(+), 3 deletions(-) diff --git a/crates/proof-topic-sql-guard/src/lib.rs b/crates/proof-topic-sql-guard/src/lib.rs index 1b485295e..56d545224 100644 --- a/crates/proof-topic-sql-guard/src/lib.rs +++ b/crates/proof-topic-sql-guard/src/lib.rs @@ -916,16 +916,45 @@ pub fn has_word(haystack: &str, needle: &str) -> bool { } /// Identifier-shaped tokens, lower-cased, dots kept (`schema.table`). +/// +/// A **double-quoted** run is one identifier, kept whole. That matters for a +/// hyphenated topic: `"fixture-topic-v0_scratch"` is a single legal identifier, +/// and splitting it at the `-` would yield `fixture`, `topic`, `v0_scratch` — +/// none of which is inside the topic's namespace, so a legal quoted name would +/// be refused as unscoped. Quotes are stripped on the way in, so the token is +/// the name the database would store and the deny rules match it as before. fn tokens(text: &str) -> Vec { let mut out = Vec::new(); let mut cur = String::new(); - for c in text.chars() { + let chars: Vec = text.chars().collect(); + let mut i = 0usize; + while i < chars.len() { + let c = chars[i]; + if c == '"' { + // A quoted identifier: its body is one token, hyphens included. + i += 1; + while i < chars.len() { + if chars[i] == '"' { + if chars.get(i + 1) == Some(&'"') { + cur.push('"'); // `""` is an escaped quote inside the name + i += 2; + continue; + } + i += 1; + break; + } + cur.push(chars[i]); + i += 1; + } + continue; + } if c.is_alphanumeric() || c == '_' || c == '.' { cur.push(c); } else if !cur.is_empty() { out.push(cur.trim_matches('.').to_ascii_lowercase()); cur.clear(); } + i += 1; } if !cur.is_empty() { out.push(cur.trim_matches('.').to_ascii_lowercase()); diff --git a/crates/proof-topic-sql-guard/tests/sql_guard.rs b/crates/proof-topic-sql-guard/tests/sql_guard.rs index 84c414d31..fe239ce9d 100644 --- a/crates/proof-topic-sql-guard/tests/sql_guard.rs +++ b/crates/proof-topic-sql-guard/tests/sql_guard.rs @@ -14,8 +14,8 @@ use proof_topic_sql_guard::MigrationDenied; use proof_topic_sql_guard::{ - blank_statements, check_migration, is_topic_scoped, split_statements, topic_sql_prefix, - OWNED_TABLES, + blank_statements, check_migration, is_topic_scoped, referenced_objects, split_statements, + topic_sql_prefix, OWNED_TABLES, }; const TOPIC: &str = "tb4"; @@ -344,6 +344,48 @@ fn allowed_for(sql: &str, topic: &str) { .unwrap_or_else(|e| panic!("{sql:?} must be allowed for {topic}: {e}")); } +/// The **quoted** literal spelling is usable too. +/// +/// `"fixture-topic-v0_scratch"` is one legal identifier, and `is_topic_scoped` +/// accepts it. It is only reachable if the tokenizer keeps a quoted run whole: +/// splitting at the `-` yields `fixture` / `topic` / `v0_scratch`, none of which +/// is inside the topic's namespace, so a legal quoted name would be refused as +/// unscoped. +#[test] +fn a_quoted_hyphenated_name_is_one_identifier() { + let topic = "fixture-topic-v0"; + allowed_for( + r#"CREATE TABLE "fixture-topic-v0_scratch" (id TEXT)"#, + topic, + ); + allowed_for(r#"CREATE TABLE "fixture-topic-v0".runs (id TEXT)"#, topic); + allowed_for(r#"SELECT id FROM "fixture-topic-v0_scratch""#, topic); + + // Quoting is not an escape hatch: a sibling, a `topic_*` name, and a + // `proof_*` object are still refused when quoted. + for sql in [ + r#"CREATE TABLE "fixture-topic-v1_scratch" (id TEXT)"#, + r#"CREATE TABLE "topic_scratch" (id TEXT)"#, + r#"DROP TABLE "proof_rule_version""#, + ] { + assert!( + check_migration(sql, topic).is_err(), + "{sql:?} must be refused for {topic}" + ); + } +} + +/// A quoted identifier with an escaped quote still reads as one name. +#[test] +fn an_escaped_quote_inside_a_quoted_name_is_kept() { + // `""` inside a quoted identifier is one literal quote in the name. + let names = referenced_objects(r#"CREATE TABLE "a""b" (id TEXT)"#); + assert!( + names.iter().any(|t| t == "a\"b"), + "the doubled quote is one name, got {names:?}" + ); +} + /// A quoted identifier still names an object, so quoting cannot smuggle a /// denied table past the scanner. #[test] From 0c1cf1dbe7428d515b73305b6c0aaf7875c14065 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 19:40:31 +0000 Subject: [PATCH 12/42] feat(proof-admin): topic lifecycle makes a long drive legible MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `topic install --drive-rlm` prints one line and then nothing until the whole run returns. Provisioning a VM, the RLM's `propose_rules` job, and a paid baseline can legitimately take hours, so an operator watching a working run sees exactly the same output as one watching a run that died — and the first question a stalled run raises ("why are there still 0 rows in `proof_baseline_measurement`?") cannot be answered from the CLI at all. `proof-admin topic lifecycle ` reads the durable progress: the newest `proof_lifecycle_event` transition, the rule version in force and its provenance, whether a baseline exists and under which version, the full transition history, and what to do next read off the state. Read-only — it never writes, never moves the lifecycle, and never spends. The states it names are the ones that actually stall: `provisioning` (the VM is being created; re-run the same command to resume if no CLI is attached) and `baselining` with no baseline row (the RLM's `propose_rules` → `baseline` jobs run there, and a paid baseline can take hours). A topic with no transitions is an error that says nothing has driven it, not an empty success an operator could misread as healthy. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- bins/proof-admin/src/main.rs | 60 ++++++++++++++ bins/proof-admin/tests/cli.rs | 111 ++++++++++++++++++++++++++ crates/proof-topic-ops/src/lib.rs | 2 +- crates/proof-topic-ops/src/seal.rs | 121 +++++++++++++++++++++++++++++ 4 files changed, 293 insertions(+), 1 deletion(-) diff --git a/bins/proof-admin/src/main.rs b/bins/proof-admin/src/main.rs index 53371b0d0..4dd8ce33b 100644 --- a/bins/proof-admin/src/main.rs +++ b/bins/proof-admin/src/main.rs @@ -179,6 +179,18 @@ enum TopicCmd { }, /// List installed topics: a read-only view of `proof_topic_version`. List, + /// Where a topic is in its RLM lifecycle, and what it is waiting on. + /// + /// Read-only. `topic install --drive-rlm` prints one line and then nothing + /// until the whole run returns — provisioning a VM, the RLM's + /// `propose_rules` job, and a paid baseline can take hours, so this is the + /// read that tells a working run from a stopped one. The durable progress + /// is `proof_lifecycle_event`; a run that died left its last transition + /// here. + Lifecycle { + /// Topic slug, or an alias of one. + topic_id: String, + }, /// Show one topic's newest install: the `proof_topic_install` journal. InstallLog { /// Topic slug. @@ -382,6 +394,7 @@ async fn run_topic(opts: &Options, cmd: &TopicCmd) -> Result<(), Failure> { } TopicCmd::List => cmd_list(opts).await, TopicCmd::InstallLog { topic } => cmd_install_log(opts, topic).await, + TopicCmd::Lifecycle { topic_id } => cmd_lifecycle(opts, topic_id).await, TopicCmd::Show { topic_id } => cmd_show(opts, topic_id).await, TopicCmd::Alias { cmd } => run_alias(opts, cmd).await, TopicCmd::Disable { @@ -616,6 +629,53 @@ async fn cmd_install_log(opts: &Options, topic_id: &str) -> Result<(), Failure> install::install_log(opts, topic_id).await } +/// `topic lifecycle`: read the journal and the provenance, and say what is next. +async fn cmd_lifecycle(opts: &Options, topic_id: &str) -> Result<(), Failure> { + let pool = open_pool(opts).await?; + let report = proof_topic_ops::lifecycle(&pool, topic_id) + .await + .map_err(ops_to_failure)?; + if opts.json { + return print_json(&serde_json::json!({ + "topic_id": report.topic_id, + "state": report.state, + "rules_version": report.rules_version, + "rules_source": report.rules_source, + "baseline_rules_version": report.baseline_rules_version, + "history": report.history, + "next": report.next_steps(), + })); + } + println!("topic {} — lifecycle", report.topic_id); + println!(" state {}", report.state); + println!( + " rules_version {}", + report + .rules_version + .map_or_else(|| "-".to_owned(), |v| v.to_string()) + ); + println!( + " rules_source {}", + dash_if_empty(report.rules_source.as_deref().unwrap_or_default()) + ); + println!( + " baseline {}", + report.baseline_rules_version.map_or_else( + || "not measured".to_owned(), + |v| format!("measured (rules v{v})") + ) + ); + if !report.history.is_empty() { + println!(" history (oldest first)"); + for line in &report.history { + println!(" {line}"); + } + } + println!(); + println!("{}", report.next_steps()); + Ok(()) +} + /// `topic baseline`: read the measurement and print what to seal. /// /// The procedure is [`proof_topic_ops::baseline`]; this is the printing. diff --git a/bins/proof-admin/tests/cli.rs b/bins/proof-admin/tests/cli.rs index 08e6b5121..0a080246c 100644 --- a/bins/proof-admin/tests/cli.rs +++ b/bins/proof-admin/tests/cli.rs @@ -962,6 +962,7 @@ fn read_commands_without_a_database_are_usage_errors() { for args in [ vec!["topic", "list"], vec!["topic", "show", "fixture-topic-v0"], + vec!["topic", "lifecycle", "fixture-topic-v0"], ] { let out = run(&args); assert_eq!(code(&out), EXIT_USAGE, "{args:?}: {}", stderr(&out)); @@ -1073,6 +1074,7 @@ fn help_lists_every_subcommand() { "validate", "install", "install-log", + "lifecycle", "list", "show", "enable", @@ -1221,6 +1223,115 @@ async fn the_registry_view_lists_what_the_scoring_path_persisted() { tp.drop_schema().await.expect("drop"); } +/// `topic lifecycle` reports where a driven topic actually is. +/// +/// This is the read that makes a long `--drive-rlm` legible: the command +/// prints one line and then nothing until the whole run returns, so an +/// operator watching a working run sees the same output as one watching a +/// stopped run. The durable progress is the lifecycle journal, and the +/// baseline's absence is what says the paid job has not landed yet — the +/// exact question "0 rows in `proof_baseline_measurement`" raises. +#[tokio::test] +async fn the_lifecycle_view_says_where_a_driven_topic_is() { + use proof_rlm::{RlmEvent, RlmState}; + use proof_rlm_store::{RlmStore, TransitionRow}; + + let Some(url) = std::env::var("DATABASE_URL") + .ok() + .map(|u| u.trim().to_owned()) + .filter(|u| !u.is_empty()) + else { + return; + }; + let tp = match db::test_pool_with_url(&url).await { + Ok(tp) => tp, + Err(e) => panic!("test_pool: {e}"), + }; + let store = proof_rlm_store::PgRlmStore::new(tp.pool().clone()); + let doc = fixture::signed_topic(&format!("sha256:{}", "ab".repeat(32))); + RlmStore::put_topic_version(&store, &doc) + .await + .expect("persist the document"); + + let schema = tp.schema().to_owned(); + let scoped = format!("{url}?options=-c%20search_path%3D{schema}%2Cpublic"); + let run_db = |args: &[&str]| { + Command::new(env!("CARGO_BIN_EXE_proof-admin")) + .args(args) + .env("BASE_DATABASE_URL", &scoped) + .env_remove("BASE_DATABASE_URL_FILE") + .output() + .expect("run proof-admin") + }; + + // No transitions yet: an error that says nothing has driven it, not an + // empty success an operator could misread as "fine". + let out = run_db(&["topic", "lifecycle", "fixture-topic-v0"]); + assert_eq!(code(&out), EXIT_ERROR, "stderr={}", stderr(&out)); + assert!( + stderr(&out).contains("no lifecycle rows"), + "{}", + stderr(&out) + ); + + // The shape a `--drive-rlm` run leaves mid-flight: past the owner gate, + // provisioning, with **no** baseline row yet. + for (from, event, to) in [ + ( + RlmState::Draft, + RlmEvent::SubmitForReview, + RlmState::OwnerPresend, + ), + ( + RlmState::OwnerPresend, + RlmEvent::OwnerApproved, + RlmState::AwaitingOwnerKeys, + ), + ( + RlmState::AwaitingOwnerKeys, + RlmEvent::OwnerKeysPresent, + RlmState::Provisioning, + ), + ] { + RlmStore::record_transition( + &store, + &TransitionRow { + topic_id: doc.id.clone(), + from, + event, + to, + note: "test".into(), + }, + ) + .await + .expect("record"); + } + + let out = run_db(&["--json", "topic", "lifecycle", "fixture-topic-v0"]); + assert_eq!(code(&out), 0, "stderr={}", stderr(&out)); + let view: serde_json::Value = serde_json::from_str(&stdout(&out)).expect("json"); + assert_eq!(view["topic_id"], "fixture-topic-v0"); + assert_eq!(view["state"], "provisioning"); + assert!( + view["baseline_rules_version"].is_null(), + "no baseline has been measured yet: {view}" + ); + assert!( + view["history"].as_array().is_some_and(|h| h.len() == 3), + "every transition is reported, oldest first: {view}" + ); + // The advice names the state, so an operator knows the run is in flight + // rather than lost. + assert!( + view["next"] + .as_str() + .is_some_and(|s| s.contains("VM is being created")), + "{view}" + ); + + tp.drop_schema().await.expect("drop"); +} + // --------------------------------------------------------------------------- // The publish order: an `open` document is not publishable before the install // --------------------------------------------------------------------------- diff --git a/crates/proof-topic-ops/src/lib.rs b/crates/proof-topic-ops/src/lib.rs index f339bf92e..0cd080d87 100644 --- a/crates/proof-topic-ops/src/lib.rs +++ b/crates/proof-topic-ops/src/lib.rs @@ -34,7 +34,7 @@ pub mod seal; pub use drive::{drive, DriveOutcome}; pub use publish::PublishTarget; -pub use seal::{baseline, seal, BaselineReport, SealArgs, SealOutcome}; +pub use seal::{baseline, lifecycle, seal, BaselineReport, LifecycleReport, SealArgs, SealOutcome}; /// Why an operator procedure refused. /// diff --git a/crates/proof-topic-ops/src/seal.rs b/crates/proof-topic-ops/src/seal.rs index 883959567..8ed85f16f 100644 --- a/crates/proof-topic-ops/src/seal.rs +++ b/crates/proof-topic-ops/src/seal.rs @@ -401,6 +401,127 @@ fn next_seal_steps(topic_id: &str, commitment: &str) -> String { ) } +/// Where a topic is in its RLM lifecycle, and what it is waiting on. +/// +/// `topic install --drive-rlm` prints one line and then **nothing** until the +/// whole run returns: provisioning a VM, the RLM's `propose_rules` job, and a +/// paid baseline can legitimately take hours, and a run that is working is +/// indistinguishable from one that is stuck if the only observable is "no +/// output yet". The durable progress is the lifecycle journal, so this is the +/// read that makes a long run legible — and, when a run dies, the last +/// transition is what says how far it got. +#[derive(Debug, Clone, PartialEq)] +pub struct LifecycleReport { + /// Canonical topic slug (the alias resolved). + pub topic_id: String, + /// The state the newest transition left the topic in. + pub state: String, + /// Rule version in force, when the RLM has written one. + pub rules_version: Option, + /// Whether that version is RLM-authored (`proof_rule_version.source`). + pub rules_source: Option, + /// Whether a baseline has been measured (and under which version). + pub baseline_rules_version: Option, + /// Every transition, oldest first: `from -> to (event)`. + pub history: Vec, +} + +impl LifecycleReport { + /// What the operator should do next, read off the state. + #[must_use] + pub fn next_steps(&self) -> String { + match self.state.as_str() { + "draft" => "Nothing has run yet. Drive the RLM: `proof-admin topic install \ + --bundle --env --drive-rlm --owner-approved`." + .to_owned(), + "owner_presend" | "awaiting_owner_keys" => { + "The lifecycle is waiting on the owner (approval, then the owner key file). \ + `--drive-rlm` needs `--owner-approved` and \ + `PROOF_RLM_OWNER_INFERENCE_KEY_FILE` present." + .to_owned() + } + "provisioning" => "The VM is being created. A `--drive-rlm` run is in flight if the \ + CLI is still attached; if it is not, this state is where it \ + stopped — re-run the same command to resume." + .to_owned(), + "baselining" => { + if self.baseline_rules_version.is_some() { + "A baseline is measured. Seal it: `proof-admin topic baseline \ + ` then `topic seal … --publish`." + .to_owned() + } else { + "No baseline yet. The RLM's `propose_rules` → `baseline` jobs run here; a \ + paid baseline can take hours. If no CLI is attached, the run stopped — \ + re-run `topic install --drive-rlm --owner-approved` to resume from this \ + state." + .to_owned() + } + } + "open" => "The topic is open. Miners can submit; confirm `can_score` on \ + `GET /v1/status`." + .to_owned(), + other => format!("State {other:?} is not one this command gives advice for."), + } + } +} + +/// `topic lifecycle`: read the journal and the provenance, and say what is next. +/// +/// Read-only: it never writes, never moves the lifecycle, and never spends. +/// +/// # Errors +/// +/// [`OpsError::error`] when the topic is unknown or the store cannot be read. +pub async fn lifecycle(pool: &sqlx::PgPool, topic_id: &str) -> Result { + let store = PgRlmStore::new(pool.clone()); + let (canonical, _, _) = resolve_topic(&store, topic_id).await?; + let lc = store + .lifecycle(&canonical) + .await + .map_err(|e| OpsError::error(format!("{canonical} lifecycle: {e}")))? + .ok_or_else(|| { + OpsError::error(format!( + "topic {canonical:?} has no lifecycle rows: nothing has driven it yet. \ + `proof-admin topic install --drive-rlm --owner-approved` writes the first one." + )) + })?; + let rules_version = store + .current_rules(&canonical) + .await + .map_err(|e| OpsError::error(format!("{canonical} rules: {e}")))? + .map(|r| r.version); + let rules_source = store + .current_rules_source(&canonical) + .await + .map_err(|e| OpsError::error(format!("{canonical} rule provenance: {e}")))? + .map(|s| format!("{s:?}").to_lowercase()); + let baseline_rules_version = store + .baseline(&canonical) + .await + .map_err(|e| OpsError::error(format!("{canonical} baseline: {e}")))? + .map(|b| b.rules_version); + let history = lc + .history + .iter() + .map(|t| { + format!( + "{} -> {} ({})", + t.from.as_str(), + t.to.as_str(), + t.event.as_str() + ) + }) + .collect(); + Ok(LifecycleReport { + topic_id: canonical, + state: lc.state.as_str().to_owned(), + rules_version, + rules_source, + baseline_rules_version, + history, + }) +} + /// What to do once the topic is open. fn after_seal(topic_id: &str, published: bool) -> String { if published { From b2c8fe9e9227918b766952eb67a6c71fbdf94072 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Tue, 15 Sep 2026 19:46:50 +0000 Subject: [PATCH 13/42] fix(guest): an unresolved task_slice fails closed, always MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit LIVE Gate 1 root cause. `task_slice=tb4-first-5` on a pack whose only slice-ish file was `MANIFEST_FIRST15` (no `slices/`) resolved to nothing, and `select_base` treated that as *informational* when the pack defined no slices — falling through to the next rule and scoring **every** task the pack held. A topic that named 5 tasks was scored on 10, the run overran its 14400 s wall clock, and no baseline was measured at all. The label is the topic's assertion about *which* tasks to score, so the guest must never silently widen it. An unresolved label is now a refusal whether or not the pack defines any other slice, and the refusal names what the pack does define (the labels and their files, or plainly that it has none) plus the two ways forward: name the set with `params.tasks`, or add the slice to the pack. `task_count` is now accepted as a **legacy alias for `n_tasks`**, used only when `n_tasks` is absent (`n_tasks` wins). It is a count, never a selector: it cannot stand in for a slice, so a topic that sets both an unresolvable slice and a count is refused rather than scored on an arbitrary N. Tests: the informational case is replaced by three fail-closed cases (no slices / typo against a pack with slices / never widens to `allow` or the whole pack), plus the `task_count` alias and the property that a count cannot substitute for a slice. Proven non-vacuous: with the old escape restored, 3 tests fail. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- deploy/guest/runners/README.md | 2 +- .../runners/rlm_fc_in_guest_harbor/README.md | 4 +- .../harness/filter_tasks.py | 90 ++++++++++++-- .../runners/rlm_fc_in_guest_harbor/lib.sh | 4 + .../tests/test_filter_tasks.py | 117 +++++++++++++++++- 5 files changed, 198 insertions(+), 19 deletions(-) diff --git a/deploy/guest/runners/README.md b/deploy/guest/runners/README.md index a7a40aea2..34c632292 100644 --- a/deploy/guest/runners/README.md +++ b/deploy/guest/runners/README.md @@ -81,7 +81,7 @@ closed on an unmeasured one). |-------|-------|------------------------| | `tasks` | `[A-Za-z0-9][A-Za-z0-9_.-]{0,63}` names, comma / space separated | The exact items to score, in order. An item the pack does not hold **fails closed** (never a smaller set). One name = the single-task smoke | | `task_exclude` | same list shape | Items never scored | -| `n_tasks` | positive integer | Keep only the first N selected items (`1` = smoke) | +| `n_tasks` | positive integer | Keep only the first N selected items (`1` = smoke). `task_count` is a legacy alias, used only when `n_tasks` is absent; it is a **count, never a selector** and cannot stand in for a slice | | `max_task_duration_s` | positive integer | Drop items whose **known** duration is at or over this; absent = no gate | | `exclude_unknown_duration` | `"true"` / `"false"` | Under a gate, drop items with no duration metadata | | `exec_timeout_s` | positive integer | Default wall clock for one command the miner's harness runs without its own timeout (the reference adaptor exports it as `PROOF_EXEC_TIMEOUT_S`) | diff --git a/deploy/guest/runners/rlm_fc_in_guest_harbor/README.md b/deploy/guest/runners/rlm_fc_in_guest_harbor/README.md index 9a64714f5..71196d609 100644 --- a/deploy/guest/runners/rlm_fc_in_guest_harbor/README.md +++ b/deploy/guest/runners/rlm_fc_in_guest_harbor/README.md @@ -126,12 +126,12 @@ Selection, in order, then filters: | Step | Source | Behaviour | |------|--------|-----------| | 1 | `params.tasks` (`PROOF_PARAM_TASKS`) | Exact ordered names (comma / space separated). A name the pack does not hold **fails closed** — a topic that names a task is never scored on a smaller set. **One name is the single-task smoke.** | -| 2 | `constraints.task_slice` (`PROOF_TASK_SLICE`) | Resolved **through the pack**: `slices/