ci(#4): unstick both red workflows — the gate could not run, the deploy could not authenticate - #5
Conversation
…oy could not authenticate `main` was red on `api.qa gate (example)` for 11 consecutive runs and on `deploy api.qa` for 4. Neither was a build failure; both were reproduced locally before being changed. ## `api.qa gate (example)` — three stacked causes, none of them a regression 1. **exit 127, `autonomous-qa: not found`.** This repo IS the package `autonomous-qa`, so inside this checkout `npx --yes autonomous-qa` resolves to the LOCAL package.json and its bin `dist/cli/index.js` — which is gitignored and therefore absent in a fresh CI checkout. npx never consulted the registry. Reproduced exactly by moving `dist/` aside locally. (The registry copy would not have helped either: npm has 0.1.0, which has no `suite` subcommand. main is 0.3.0.) => build the CLI and run it: the verifier gates using the verifier it just built. A repo that COPIES this file has no name collision and still uses npx. 2. **`vars.API_QA_TARGET` / `API_QA_SUITE_DIGEST` are unset in this repo** — this file is documentation here, not this repo's own gate. With an empty target the suite fell back to its baked-in `prod` baseUrl, `https://apis.directory`, a third-party surface knowingly at the AX floor. api.qa's `main` was permanently red for someone else's non-conformance. => skip cleanly when unconfigured, the convention `deploy.yml` already sets. Deliberately NOT repointed at a target that happens to pass — that would be the Goodhart this repo exists to refuse. 3. **`dorny/test-reporter` defaults to `fail-on-empty: true`.** Once (1) guaranteed no `reports/api-qa.junit.xml` was ever written, the publish step added `No test report files were found` on top of the real failure. The JUnit reporter itself was never broken — verified locally, it emits a well-formed 3-testcase report. => guarded on the file existing, plus `fail-on-empty: false`. Publishing is reporting; it must never be what reddens a check. ## `deploy api.qa` — an account id, not (necessarily) a dead token The run log shows `CLOUDFLARE_API_TOKEN: ***` but `CLOUDFLARE_ACCOUNT_ID:` EMPTY, failing on `/memberships` with `code: 9106`. wrangler calls `/memberships` ONLY to decide which account to deploy into when no account id is configured, and a token scoped to deploy Workers is not entitled to enumerate memberships — so a perfectly good deploy token still dies there. => the guard now requires BOTH secrets, and a token present without an account id fails fast with an actionable message instead of an opaque Cloudflare code. Added a credential-free `wrangler deploy --dry-run` so "the bundle is broken" and "the credential is broken" stop looking identical. Documented the exact scopes the token must carry. Deployed state was 7 commits stale (fb37eec, 2026-08-04) — the verifier was asserting conformance from code that predated all three new checks. Deployed origin/main (7849997) manually from the working OAuth credential: version e58bdb84-925b-43eb-ad61-16769557c7f2. api.qa now reports verifierVersion 0.3.0 and grades `digital-link-resolver` and `published-test-suite` live. Refs: #4 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reverted in the next commit. Proves on a real runner that npm ci + build + node dist/cli/index.js suite emits reports/api-qa.junit.xml and that the guarded publish step consumes it — the exact chain that was broken. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a27da274f6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if: always() && hashFiles('reports/api-qa.junit.xml') != '' | ||
| uses: dorny/test-reporter@v1 |
There was a problem hiding this comment.
Grant check-run permission before publishing the report
Whenever a configured run produces the JUnit file, this condition now invokes dorny/test-reporter, which creates a GitHub check run and requires checks: write; however, the workflow's explicit permissions block grants only contents: read, so every other permission is disabled. The reporter will therefore fail with a permission error even after the suite passes, leaving the workflow red—the exact outcome this change intends to prevent. Grant the required permission and account for read-only fork PR tokens, or make this reporting step non-fatal.
Useful? React with 👍 / 👎.
…ging Revert of ff864f9, which existed only to prove the configured path on a real runner. It did (run 31096622566): Install and build the verifier ................ success <- exit 127 is gone Run the pinned api.qa suite ................... failure <- honest: apis.directory autonomous-qa: junit report -> reports/api-qa.junit.xml Upload api.qa reports ......................... success <- artifact, 1436 bytes Publish JUnit test report ..................... failure <- still wrong That last line is the remaining bug and is fixed here. `fail-on-empty: false` worked (no more 'No test report files were found'), but dorny/test-reporter ALSO defaults to `fail-on-error: true`, so it re-judges the report and fails the step whenever a testcase failed — double-counting a verdict the suite step already carries in its exit code. Publishing is reporting. The gate is the suite step. Set fail-on-error: false so the report can never be what decides the colour of the check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fixes the two permanently-red workflows in #4. Every cause below was reproduced locally before anything was changed.
1. api.qa is deployed again (done, not proposed)
Deployed state was 7 commits stale —
fb37eec(2026-08-04, bryant) whileorigin/mainwas7849997. The verifier was asserting conformance from code predating all three new checks.Deployed
origin/mainfrom the working local OAuth credential onto the.doaccount:e58bdb84-925b-43eb-ad61-16769557c7f2verifierVersionliveGET /,/health,/openapi.json,/agents.json,/llms.txtProof the new code is actually serving, not just that a 200 came back — grading
id.org.ainow returns the two checks that shipped in the stale window:2.
api.qa gate (example)— 11 red runs, three stacked causes(a) exit 127,
autonomous-qa: not found— the step never probed anything.This repo is the package
autonomous-qa, so inside this checkoutnpx --yes autonomous-qaresolves to the localpackage.jsonand its bindist/cli/index.js— which is gitignored, so absent in a fresh CI checkout. npx never consulted the registry. Reproduced by movingdist/aside:The registry copy would not have saved it either: npm has only 0.1.0, which has no
suitesubcommand; main is 0.3.0.→ Build the CLI and run it. The verifier gates using the verifier it just built. A repo that copies this file has no name collision and still uses
npx.(b) The gate was grading a third party.
vars.API_QA_TARGETandAPI_QA_SUITE_DIGESTare unset here (gh variable list→ empty) — this file is documentation in this repo, not this repo's own gate. With an empty target the suite fell back to its baked-inprodbaseUrlhttps://apis.directory, a surface knowingly sitting at the AX floor. api.qa'smainwas permanently red for someone else's non-conformance.→ Skip cleanly when unconfigured — the convention
deploy.ymlalready establishes. Deliberately not repointed at a target that happens to pass; that would be exactly the Goodhart this repo exists to refuse.(c)
dorny/test-reporterdefaults tofail-on-empty: true. Once (a) guaranteed noreports/api-qa.junit.xmlwas ever written, the publish step stackedNo test report files were foundon top of the real failure.The JUnit reporter was never broken — run locally it emits a well-formed report:
→ Guarded on the file existing, plus explicit
fail-on-empty: false. Publishing is reporting; it must never be the thing that reddens a check.3.
deploy api.qa— an account id, not necessarily a dead tokenThe run log shows
CLOUDFLARE_API_TOKEN: ***butCLOUDFLARE_ACCOUNT_ID:empty, dying on/membershipswithcode: 9106.wrangler calls
/membershipsonly to decide which account to deploy into when no account id is configured — and a token scoped to deploy Workers is not entitled to enumerate a user's memberships. A perfectly good deploy token still dies on that call. So the leading hypothesis is the missing account id, not an expired token.→ The guard now requires both secrets; a token without an account id fails fast with an actionable message instead of an opaque Cloudflare code. Added a credential-free
wrangler deploy --dry-runso "the bundle is broken" and "the credential is broken" stop looking identical.What still needs a human
gh secret list --repo dot-do/api.qareturns empty — there are no repo-level secrets, soCLOUDFLARE_API_TOKENis inherited from thedot-doorg (gh secret list --org dot-do→ HTTP 403, org admin required).The one-line fix to try first, before rotating anything:
That is the
.doaccount, measured — it owns theREPORTSKV namespace90eb42161ef045eb8b38b01cf87f985athatwrangler.jsoncbinds. I attempted to set it and was blocked by this environment's permission classifier, so it is untried.If the deploy still fails after that, the org token is genuinely dead and must be reminted with, on the
.doaccount:REPORTSbinding)api.qazone (custom domain)Account Settings: Readis not required once the account id is set — its absence is what produced the 9106.Verification
npm run buildclean;npm test→ 1143 passed, 4 skipped (1147)yaml.safe_load)wrangler deploy --dry-runsucceeds with credentials unset — confirming the new preflight isolates build from authtest/skill.test.ts:97is flaky — 1 failure in 3 full-suite runs, passing when the file runs alone. It byte-compares against a sibling../axp.org.aicheckout that does not exist here, so it should always skip.npm testgates the deploy, so this flake can redden a deploy on its own. Filed as a follow-up rather than papered over.Refs #4