From 55036039afb9092033395514bd1980ff5d488170 Mon Sep 17 00:00:00 2001 From: abrichr Date: Thu, 3 Sep 2026 16:59:52 -0400 Subject: [PATCH] docs(reward): state the checks that run, and that the worker is unreleased Two classes of claim on docs.openadapt.ai/commercial/seal-reward/ are false against the published packages. The install path. The page opens "Run it locally with MockMed" with `pip install 'openadapt-flow[reward]'` and `openadapt-flow serve-reward`. openadapt-flow 1.34.0 on PyPI declares no `reward` extra, ships no `openadapt_flow/reward/`, and registers no `serve-reward` subcommand. Both of those exist only at the repository head. Anyone following the page today gets a failed install and an unknown command. The published commands stay as what the release will carry, with the git-head install beside them. Enforcement that does not exist. Nothing verifies a reward certificate's signature: `_validate_signature` in openadapt_types checks base64 and a 64-byte length, and there is no issuer key registry. `issuer` is an unauthenticated string; `RewardCertificateV1._issue_window` refuses only `self_signed` combined with a non-synthetic scope, so an `organization` issuer with `production` scope validates. `grep -rni revok` across the reward code in openadapt-types, openadapt-evals and openadapt-flow returns nothing. `RewardCertificateV1.satisfies` has no caller outside a unit test. So this drops the revocation column and the revocation-key gloss, restates `issuer` as a declared field, says plainly that nothing calls `satisfies`, and replaces "the only issuer that can state production scope" with what the design intends and what the code actually reads. The properties that are enforced stay as they were: the tier floor (`refuse_development_certification`, `score`), the unscored rule (`RewardScoringPolicyV1.scalar_for`, `_scoring_contract`), receipt-versus- Seal separation, idempotency and digest binding (`RewardWorker.score_episode`, `RewardWorker._check_binding`), and credentials staying on the worker's machine. Co-Authored-By: Claude Opus 5 --- docs/commercial/seal-reward.md | 32 +++++++++++++++++++++++--------- 1 file changed, 23 insertions(+), 9 deletions(-) diff --git a/docs/commercial/seal-reward.md b/docs/commercial/seal-reward.md index e5adb0c..312dd9b 100644 --- a/docs/commercial/seal-reward.md +++ b/docs/commercial/seal-reward.md @@ -20,7 +20,7 @@ The pure scorer is `openadapt_types.score`. | Node | Runs | Holds | |---|---|---| | Organization worker (inside the customer network) | The reward worker, the oracle read, the MockMed or real system of record | Records, the oracle recipe, the calibration corpus, evidence bytes. None of these leave. | -| OpenAdapt control service (off the high-volume path) | Contract registry, certificate issue, revocation | Contracts, certificates, the revocation list, privacy-safe receipts | +| OpenAdapt control service (off the high-volume path) | Contract registry and certificate issue | Contracts, certificates, privacy-safe receipts | | Trainer node | The policy, rollouts, the optimizer | The checkpoint, episode ids, the receipts it fetched | The trainer sees ids, digests, a tier, an outcome, a scalar, and a certificate @@ -44,8 +44,8 @@ a scalar exists. `certified` on a receipt is a separate column from the scalar. It is true only when all four hold: oracle tier 2 or 3, a certificate that is current at that policy update, a `calibration_corpus_digest`, and a `calibration_scope`. The -scope is `synthetic` or `production`. Today the only certificate anyone can -compute is `synthetic` scope, calibrated on MockMed and ExtraDup. A +scope is `synthetic` or `production`. Today the only certificate our code +issues is `synthetic` scope, calibrated on MockMed and ExtraDup. A `production` scope needs the Phase-1 calibration on a held-out corpus, which is not published. Show the scope beside the word certified; `production_certified` on the receipt is that check. @@ -254,7 +254,7 @@ rate. Its fields: | Field | Meaning | |---|---| -| `certificate_id` | Revocation key. The issuer checks the revocation list, as for every other admission. | +| `certificate_id` | Names the certificate. A receipt that references a certificate carries this id. | | `reward_contract_digest` | The `RewardContractV1` this bound applies to | | `checker_configuration_digest` | The checker configuration the bound was calibrated for | | `epsilon` | Upper bound on P(false-accept) | @@ -264,7 +264,7 @@ rate. Its fields: | `calibration_scope` | `synthetic` or `production`. What corpus family the bound was calibrated against. | | `issued_at_policy_update` | The policy update the certificate was issued at | | `expiry_policy_updates` | How many policy updates it stays current | -| `issuer` | `self_signed` or `organization`. A self-signed certificate may carry only `synthetic` scope; the validator refuses the other combination. | +| `issuer` | `self_signed` or `organization`, as declared by whoever built the certificate. The validator refuses `self_signed` with any scope but `synthetic`. It does not check who the issuer is. | | `issued_at`, `issuer_key_id`, `signature` | Ed25519 signature over the unsigned payload | Expiry counts policy updates, not hours. A certificate issued at update `i` @@ -277,13 +277,16 @@ every receipt after it, and a certified arm halts. `RewardContractV1.certificate_policy` states the weakest certificate the contract accepts (`epsilon`, `delta`, `threshold`, corpus digest, expiry). -`RewardCertificateV1.satisfies(policy)` is the check. +`RewardCertificateV1.satisfies(policy)` compares the two. Nothing on the +worker or trainer path calls it, so a receipt is not refused today for a +certificate weaker than its contract asks. The MockMed worker signs its own certificate, so every certificate it issues is `self_signed` and `synthetic`. That is enough to prove the plumbing and to -bound a synthetic run. It says nothing about a production checker. An -`organization` issuer holds the calibration corpus and the signing key, and -it is the only issuer that can state `production` scope. +bound a synthetic run. It says nothing about a production checker. A +`production` scope is meant to come from an `organization` issuer that holds +the calibration corpus and the signing key. That issuer does not exist yet, +and no code reads `issuer` as anything but a declared string. The re-certification cadence, the vacuity check, and the kill criteria are registered in the public @@ -292,11 +295,22 @@ registered in the public ## Run it locally with MockMed +The reward worker is not in a published `openadapt-flow` release yet, and +the release that carries it has no date. When it lands, two commands start +the MockMed worker: + ```bash pip install 'openadapt-flow[reward]' openadapt-flow serve-reward --seed-mockmed --port 8788 ``` +Until then, install the repository head: + +```bash +uv pip install "openadapt-flow[reward] @ git+https://github.com/OpenAdaptAI/openadapt-flow@main" +openadapt-flow serve-reward --seed-mockmed --port 8788 +``` + `--seed-mockmed` writes two contract bundles and their fixtures under the data directory and serves the tier-2 one when `--contract` is omitted. `contracts/mockmed` reads `mockmed/records.json` through the `json_file`