From 1bc330814c0657546a90c33b64687e1946358302 Mon Sep 17 00:00:00 2001 From: Felix Stubner Date: Fri, 11 Sep 2026 17:05:53 +0100 Subject: [PATCH] fix(release): serialise publishing against release-drafter release-drafter rewrites its draft by listing every release, taking the first one whose `draft` is true, then PATCHing that release by id. It re-checks nothing between the read and the write, and the payload it sends always carries `draft: true`. A publish landing in that gap is overwritten. Run 34597984827 built its payload at 12:15:46.96 and logged "Release updated!" at 12:15:47.96. v0.3.1 was published at 12:15:47Z, one second inside the window. The release came back with its tag replaced by an `untagged-...` placeholder and had to be repaired by hand. Nothing failed, and no workflow reported anything wrong -- it was found by chance, on a release that was already public. The collision is structural rather than unlucky: "merge the changelog, then publish" puts a push to main and a publish back to back, and the push is exactly what starts a drafter run. The gap is inside the action, so it cannot be closed from outside. Serialise instead: - Add publish-release.yml, a workflow_dispatch that validates the tag shape, refuses a missing or already-published release, flips the draft, then reads it back to confirm the release is public and still carries its tag. - Give it and release-drafter.yml a shared `release-draft` concurrency group, so a publish and a draft rewrite can never overlap. release-drafter was the only workflow in the repo without a concurrency group, so this also stops two drafter runs racing when several PRs merge together. - `cancel-in-progress: false` on both sides, which is load-bearing: the setting is read from the workflow of the run joining the group, so a cancelling drafter would kill an in-progress publish and strand a public release with no pipeline started. publish-release.yml dispatches release.yml and publish.yml explicitly, because GitHub suppresses `release: published` for anything done with GITHUB_TOKEN and `workflow_dispatch` is one of its two documented exceptions. That also removes an existing race, where release.yml and publish.yml both fired on the same event at the same instant. This only covers publishes that go through the workflow. `gh release edit --draft=false` and the web UI's publish button hold no lock, so docs/RELEASE.md and docs/PUBLISHING.md now point at the workflow, and publish.yml's trigger comment no longer calls the event path canonical. Unverified, and left unclaimed in the comments: why the clobber replaced the tag specifically. The PATCH demonstrably sent `tag_name: v0.3.1`, and GitHub does not expose release edit history, so the placeholder is a consequence I could not reconstruct. What is established is that the write landed on a release that was no longer a draft. --- .github/workflows/publish-release.yml | 127 ++++++++++++++++++++++++++ .github/workflows/publish.yml | 9 +- .github/workflows/release-drafter.yml | 19 ++++ docs/PUBLISHING.md | 10 +- docs/RELEASE.md | 21 ++++- 5 files changed, 176 insertions(+), 10 deletions(-) create mode 100644 .github/workflows/publish-release.yml diff --git a/.github/workflows/publish-release.yml b/.github/workflows/publish-release.yml new file mode 100644 index 00000000..e44597ed --- /dev/null +++ b/.github/workflows/publish-release.yml @@ -0,0 +1,127 @@ +name: Publish release + +# Promotes a drafted GitHub release to public. +# +# Publishing does not need a workflow. This one exists to hold a lock. +# +# release-drafter rewrites its draft by listing every release, taking the +# first one whose `draft` is true, and then PATCHing that release by id. It +# re-checks nothing between the read and the write, and the payload it sends +# always carries `draft: true` (see `find-previous-releases.ts` and +# `update-release.ts` at the SHA pinned in release-drafter.yml). Publish +# inside that gap and the PATCH lands on a release that is no longer a draft. +# +# Not theoretical. Drafter run 34597984827 built its payload at 12:15:46.96 +# and logged "Release updated!" at 12:15:47.96; v0.3.1 was published at +# 12:15:47Z, one second inside the window. The release came back out of it +# with its tag replaced by an `untagged-...` placeholder and had to be +# repaired by hand. Nothing failed and no workflow reported anything wrong -- +# the corruption was found by chance, on a release that was already public. +# +# The collision is not bad luck. "Merge the changelog, then publish" is the +# ordinary release sequence, and merging to main is exactly what starts a +# drafter run. +# +# `concurrency` below shares one group with release-drafter.yml, so a publish +# and a draft rewrite can never overlap. That only covers publishes that go +# through here: `gh release edit --draft=false` and the web UI's publish +# button hold no lock and stay exposed. docs/RELEASE.md sends you here for +# that reason. + +on: + workflow_dispatch: + inputs: + tag: + description: 'Tag of the drafted release to publish (e.g. v0.3.1)' + required: true + type: string + +# `contents: write` to flip the draft. `actions: write` to dispatch the two +# workflows at the end -- nothing here needs more. +permissions: + contents: write + actions: write + +# Shared with release-drafter.yml. This group is the entire point of the +# workflow; queueing behind an in-flight drafter run costs about 30 seconds. +# +# `cancel-in-progress: false` on BOTH sides, and that pairing is load-bearing +# rather than cautious. The setting is read from the workflow of the run that +# joins the group, so a drafter configured to cancel would cancel a publish +# run already in progress -- leaving the release public with neither +# release.yml nor publish.yml ever dispatched. Serialising instead costs a +# few idle seconds. +concurrency: + group: release-draft + cancel-in-progress: false + +jobs: + publish: + name: Promote draft to public + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - name: Promote the draft and start the pipeline + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + GH_REPO: ${{ github.repository }} + TAG: ${{ inputs.tag }} + shell: bash + run: | + set -euo pipefail + + # The same shape publish.yml enforces, for the same reason: the tag + # reaches sed replacements and commit messages downstream. + if [[ ! "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.]+)?$ ]]; then + echo "::error::refusing to publish malformed tag '$TAG'" + exit 1 + fi + + if ! gh api "repos/${GH_REPO}/git/ref/tags/${TAG}" >/dev/null 2>&1; then + echo "::error::no tag ${TAG} in this repository -- tag and push it first." + exit 1 + fi + + if ! is_draft=$(gh release view "$TAG" --json isDraft --jq '.isDraft' 2>/dev/null); then + echo "::error::no release found for ${TAG}. Draft it first." + exit 1 + fi + if [[ "$is_draft" != "true" ]]; then + echo "::error::${TAG} is already published. Nothing to promote." + exit 1 + fi + + echo "Promoting ${TAG}..." + gh release edit "$TAG" --draft=false + + # Read it back rather than assume the edit did what it said. This is + # the exact failure this workflow exists for, so it is worth one API + # call to prove the release is public AND still carries its tag. + after=$(gh release view "$TAG" --json isDraft,tagName --jq '"\(.isDraft) \(.tagName)"') + if [[ "$after" != "false ${TAG}" ]]; then + echo "::error::${TAG} did not survive the promote (isDraft/tagName = ${after}). Check for a concurrent release-drafter run." + exit 1 + fi + + # `release: published` does NOT fire here -- GitHub suppresses + # events raised by GITHUB_TOKEN. `workflow_dispatch` is one of its + # two documented exceptions, so the consumers are started + # explicitly instead. + # + # This is more deterministic than the event path it replaces, where + # release.yml and publish.yml both fire on `release: published` at + # the same instant and race. Order here is deliberate but not + # blocking: publish.yml's `wait_for_asset` polls for 15 minutes, so + # it tolerates release.yml still building, exactly as before. + gh workflow run release.yml -f tag="$TAG" + gh workflow run publish.yml -f tag="$TAG" + + { + echo "### Published ${TAG}" + echo + echo "- Release is public and still tagged \`${TAG}\`." + echo "- Dispatched \`release.yml\` (binaries + GUI installers)." + echo "- Dispatched \`publish.yml\` (package managers)." + echo + echo "Watch both in the Actions tab. Assets attach to the release as they build." + } >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 31684b7b..c53c8bf6 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -6,9 +6,12 @@ name: Publish to package managers # the others, and each is re-runnable from the Actions tab. # # Triggers: -# - On `release: published` (canonical path: flip a draft to public). -# - On `workflow_dispatch` with a tag input (re-run a specific job -# after fixing a transient failure). +# - On `workflow_dispatch` with a tag input. The canonical path: +# publish-release.yml dispatches this after promoting a draft, and it +# is also how you re-run a specific job after a transient failure. +# - On `release: published`, still honoured for a release promoted by +# hand. Note that promoting by hand races release-drafter -- see +# publish-release.yml for why that path is no longer recommended. on: release: diff --git a/.github/workflows/release-drafter.yml b/.github/workflows/release-drafter.yml index a351f159..102354a8 100644 --- a/.github/workflows/release-drafter.yml +++ b/.github/workflows/release-drafter.yml @@ -36,6 +36,25 @@ on: permissions: contents: write +# Shared with publish-release.yml, which holds the same group while it flips +# a draft to public. +# +# The action below rewrites its draft by listing every release, taking the +# first one whose `draft` is true, then PATCHing that release by id -- with +# no re-check in between and a payload that always carries `draft: true`. +# A publish landing in that gap gets overwritten. It happened: run +# 34597984827 wrote to v0.3.1 one second after it went public, and the +# release came back with its tag replaced by an `untagged-...` placeholder. +# Nothing failed and nothing reported it. +# +# `cancel-in-progress: false` matters on both sides. The setting comes from +# the workflow of the run joining the group, so cancelling here would kill an +# in-progress publish and strand a public release with no pipeline started. +# Serialising drafter runs instead costs a few seconds -- they take ~30. +concurrency: + group: release-draft + cancel-in-progress: false + jobs: update_release_draft: runs-on: ubuntu-latest diff --git a/docs/PUBLISHING.md b/docs/PUBLISHING.md index cf4c8360..b1d0d616 100644 --- a/docs/PUBLISHING.md +++ b/docs/PUBLISHING.md @@ -239,13 +239,17 @@ the bottom of the file. ## GitHub Releases Platform installers, and the download links the install scripts resolve, -come from the GitHub release — not crates.io. `release.yml` fires on -`release: published`. +come from the GitHub release — not crates.io. `release.yml` is started by +`publish-release.yml`, and still answers `release: published` and a manual +dispatch. ```bash git tag vX.Y.Z git push origin vX.Y.Z -# Then: Releases → Draft a new release → pick the tag → notes → publish. +# Then: Releases → Draft a new release → pick the tag → write the notes. +# Leave it as a draft, and promote it with the workflow — never with the +# web UI's publish button, which races release-drafter (see RELEASE.md). +gh workflow run publish-release.yml -f tag=vX.Y.Z ``` It builds and attaches, per asset, four files: the artifact, `.sha256`, diff --git a/docs/RELEASE.md b/docs/RELEASE.md index 6958b309..58b13513 100644 --- a/docs/RELEASE.md +++ b/docs/RELEASE.md @@ -117,11 +117,24 @@ Before the first automated release runs, add these secrets at 4. **Promote the draft to public.** This is the one human action that stays in the loop. ```bash - gh release edit vX.Y.Z --draft=false --repo fstubner/netscli + gh workflow run publish-release.yml -f tag=vX.Y.Z ``` - This single event fires both `release.yml` (builds binaries + GUI - installers, attaches them to the release) and `publish.yml` (fans out - to package managers). + Publish through this workflow, not `gh release edit --draft=false` and + not the web UI's publish button. `release-drafter` rewrites its draft on + every push to `main`, and it overwrites whatever release it picked up if + that release goes public mid-run — one second of overlap is enough, and + that is how v0.3.1 lost its tag. `publish-release.yml` holds a lock the + drafter shares; the other two routes hold nothing. "Merge the changelog, + then publish" puts a push and a publish back to back, so the overlap is + the normal sequence rather than bad luck. + + The workflow refuses a tag that does not exist or is already published, + flips the draft, reads it back to confirm the release is public and still + carries its tag, then starts `release.yml` (binaries + GUI installers) + and `publish.yml` (package managers). It dispatches those two explicitly + because GitHub suppresses `release: published` for anything done with a + workflow token — `workflow_dispatch` is one of the two documented + exceptions. 5. **Watch the dashboard.** From the release page or the Actions tab: - `Release` workflow: 11 CLI assets + 5 GUI installers attached. - `Publish to package managers` workflow: 9 jobs — a CLI and a GUI job