ci: drop staging deploy; prod pins on main + refresh testnet metadata

[echobt]

Summary

Fixes the three red workflows on main (41b51ac) by removing the DigitalOcean staging lane instead of repairing it, moving the digest→pin ladder onto prod, and refreshing the testnet metadata lock.

A) Staging auto-deploy is gone. The deploy-staging matrix (master / validator) is deleted from ci.yml, and deploy-staging.yml is deleted — nothing else referenced it. ci.yml is now fmt · clippy · test · deny · xtask only and touches no infrastructure, so the DO firewall 422 (must have no more than 50 rules on base-hosts) can no longer fail CI. The staging droplets and deploy/compose/env-staging.yml stay: that overlay is the testnet config local-e2e.sh builds on, and staging is now operator-driven via remote-deploy.sh. No firewall prune was invented.

B) Prod pins on every main update.

• images.yml: the update · commit staging pins job becomes record prod pins. It runs promote.sh --env prod --confirm-prod --force-prod over the freshly built GHCR digests and uploads deploy/pins/prod.json + deploy/digests/<sha>.json as the artifact prod-pins-<sha>. It no longer runs git push origin HEAD:main, which is what GH013 was rejecting (main requires a PR + Greptile review, so CI can never push a pin commit). --force-prod skips the staging-digest ladder check because no workflow writes deploy/pins/staging.json any more; without it prod would be pinned to a dead lane. The job also runs on workflow_dispatch, so an expired pin artifact can be rebuilt (see Rollback below).
• deploy-prod.yml: the "staging pins on origin/main match this SHA" gate is deleted. It now also triggers on a successful images run on main (plus the existing v*.*.* tags and workflow_dispatch), so merging to main deploys prod. Preflight is still fail-closed and got stricter in one place: the SHA must be an ancestor of origin/main, CI must be green for it (it polls, because ci and images run in parallel on a main push), and the images run for that SHA must still have a live prod-pins-<sha> artifact. The Spaces backup, firewall open/close, --build-from registry deploy, and /healthz smoke are unchanged. The old git push origin main of prod pins is replaced by an artifact — same GH013 problem.
• Rollback is a deploy-prod dispatch on the previous good commit SHA. If that commit's pin artifact has expired, preflight refuses rather than deploying something unpinned; re-run that commit's images run, or dispatch images.yml on a ref pointing at it, and the pins are rebuilt from the same GHCR digests.
• Drive-by fix in the same path: ghcr-public publicized hardcoded base/<svc> package names, which have 404'd since the repo rename (the packages are cortex/<svc>). Both copies now derive the segment from $GITHUB_REPOSITORY. This matters more now that prod pulls digests on every main update — remote-deploy.sh never logs in to GHCR, so a private package is a broken prod deploy.

C) Metadata drift. metadata/testnet.lock regenerated with cargo run -p xtask -- metadata-snapshot. The whole diff is spec_version 445 → 455 and the matching metadata_digest: a Finney testnet runtime upgrade. Every structural field we depend on is byte-identical — all five tracked call indices (set_weights 7/0, commit_timelocked_weights 7/113, commit_timelocked_mechanism_weights 7/118, set_subnet_identity, serve_axon), the WeightsTlockPayload shape, commit_reveal_version 4, all seven epoch-schedule sources, and the Commitments pallet. Nothing was blanked. Only the testnet lock exists; there is no mainnet lock to touch.

Ops follow-ups (not code)

1. Branch protection: if deploy staging master / deploy staging validator are required status checks on main, remove them — those jobs no longer exist and would block every PR as pending.
2. DO firewall: base-hosts is at the 50-rule cap from leaked ephemeral /32 CI holes. Prod deploys use the same do-firewall action, so someone should prune the stale rules by hand; this PR deliberately does not auto-prune, because a leaked CI rule and the operator's own SSH rule are both tcp/22 on a single /32 and are indistinguishable via the API.
3. GHCR visibility: after the first green images run with the corrected package names, confirm ghcr.io/cortexlm/cortex/<svc> pulls anonymously. If the packages stay private, prod hosts need a one-time docker login ghcr.io.
4. SN100 Proof metal tip is still manual by ops. CI no longer deploys DO staging; the prod tip path is GHCR digests → prod pins artifact → deploy-prod. No metal SSH deploy was invented — the repo has none.

Greptile

Every PR is reviewed by Greptile before merge. Config: .greptile/.

• Greptile has reviewed this PR; findings are fixed or answered — it found a P1 (a manual images dispatch skipped prod-pins, so the documented artifact-expiry recovery produced nothing and left deploy-prod blocked for that SHA), fixed in ab8ba57; latest review is 5/5, no outstanding issues
• If the bot was silent, I commented @greptileai review

Test plan

• cargo run -p xtask -- metadata-snapshot --check → OK (matches metadata/testnet.lock) against live wss://test.finney.opentensor.ai:443
• cargo run -p xtask -- loc-cap · consensus-lint · spec-check · design-check · external-docs-check all pass
• cargo fmt --all -- --check
• CI green on this branch (fmt · clippy · test · deny · xtask), plus CodeQL and Analyze (actions)
• Every workflow parses as YAML; every run: block passes bash -n
• Replayed the new prod-pins job logic locally on a clean tree with a synthetic digest manifest: five promote.sh --env prod --confirm-prod --force-prod calls produce a prod.json whose commit_sha and five sha256: service digests match, which is exactly what the job then asserts
• Verified the preflight artifact probe against the real API: gh api repos/CortexLM/cortex/actions/runs/<images-run>/artifacts with the select(.name == "…" and .expired == false) | length filter returns 1 for the digest artifact of 41b51ac
• deploy/scripts/assert-compose-matrix.sh — needs Docker, unavailable in this environment; unaffected by these changes and covered by the ci job.
• The workflow_run trigger cannot be exercised from a branch: GitHub only honours it from the default branch, so the first real proof is the first main push after merge.

Risk

Deploy path, no consensus/emission/signature surface; no Rust changed (git diff --stat against main is workflows, docs, and one JSON lock). Prod now deploys automatically on every green images run on main with no human approval step — that is the requested behaviour, and the fail-closed gates (SHA on main, CI green, pins artifact present, Spaces backup, /healthz smoke) all stay. deploy/pins/prod.json in git is now a template rather than the deployed state: CI derives the deployed pins per commit and keeps them in the run artifact, so promote.sh --rollback on the committed file is no longer the prod rollback path.

Naming

I did not rename BASE_* environment variables, deployed host paths, or base-*-v1 cryptographic domain tags. One clarification on GHCR package names: nothing about the pushed images changes (images.yml already derives ghcr.io/${GITHUB_REPOSITORY,,}/<svc>); only the visibility API call is fixed to name the packages that actually exist (cortex/<svc>) instead of the pre-rename base/<svc> ones, which return 404 today.

  

[cursor]

@greptileai review

[cursor]

@greptileai review

Pushed 26cc1cc — deploy-prod now accepts either artifact layout when installing the prod pins, same tolerance the deploy job already had.

[greptile-apps]

Greptile Summary

This update keeps production deployments digest-pinned while allowing a manually dispatched image workflow to recreate the production pin artifact for an older main commit. No new blocking issues were identified.

Confidence Score: 5/5

Safe to merge.

The two earlier manual artifact-recovery findings were resolved by echobt without explanation. Current workflow behavior creates production pin artifacts for manually dispatched image runs, which addresses the documented recovery path.

Files Needing Attention: None.

_{Reviews (5): Last reviewed commit: "Merge branch 'main' into cursor/drop-sta..." | Re-trigger Greptile}

[cursor]

@greptileai review

Pushed ab8ba57, which fixes the P1 you flagged. prod-pins (and ghcr-public) were gated on github.event_name == 'push' && github.ref == 'refs/heads/main', so the documented recovery — re-running images.yml for a commit whose pin artifact expired — produced no artifact and left deploy-prod permanently blocked for that SHA. Both jobs now run on anything that is not a pull_request, so a manual dispatch (on main or on a ref pointing at an older commit) rebuilds prod-pins-<sha>.

That does not widen what can reach prod: deploy-prod preflight still refuses any SHA that is not an ancestor of origin/main, still requires a green ci run for it, and still requires a live pin artifact. A pin artifact built from a stray branch dispatch simply never passes preflight. The two error messages and deploy/README.md now name both recovery routes (re-run the original run, or dispatch on a ref pointing at the commit).

The other findings in your summary — Harbor evaluation accepting failed runs, omitting completed trials, checklist rules, agent loading outside the staged artifact — are in code this PR does not touch (#266/#267). This PR is workflows, docs, and metadata/testnet.lock; git diff --stat against main shows no Rust changes.

[echobt]

@greptileai review

[echobt]

@greptileai please re-review after branch update with main (merge commit); prior summary was 5/5 on ab8ba57

[echobt]

@greptileai review

Re-ping: required Greptile Review check still IN_PROGRESS on head 5397a8ba (merge with main). CI already green. Please complete review so we can squash-merge.