ci(example): make the artifact-currency check advisory on pull requests - #27
Conversation
The worked example regenerates from three live Estonian services, and the final step diffs the regenerated project.qgz against the committed one. That step answers "has the committed example stopped describing its sources?" -- a question about the world, not about a branch. Its legend embeds facility counts, so one kindergarten opening in Tartu rewrites the file and blocks a merge that has nothing to do with it. That is what happened on 2026-09-09: PR #25 changed only the CLI, and the job went red because the upstream parcel count moved 518 -> 517. The same step had already failed on main's scheduled run two days earlier, which is the control -- the drift preceded the branch. So scope the assertion to the event rather than dropping it. On the schedule and on workflow_dispatch it stays a hard failure: that run exists to notice drift. On a pull request it reports a warning and does not block. The steps a PR is genuinely gated on are unchanged -- validate, verify, and the real-QGIS render all still ask whether the change broke the example. This is the workflow's own stated intent ("an outage upstream must not redden unrelated work") applied to data drift as well as to outages. Note that main stays red on the schedule until someone with access to the live sources reruns examples/tartu-development/pipeline.py and commits the regenerated project.qgz and run record. That is the alarm working. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DqoLGyKY5opHGjNMSerpHg
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 833841f9de
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # and the real-QGIS render -- are what a PR is actually gated on: they | ||
| # ask whether this change broke the example, not whether the example | ||
| # still matches sources that moved underneath it. | ||
| continue-on-error: ${{ github.event_name == 'pull_request' }} |
There was a problem hiding this comment.
When a PR changes pipeline.py (or supporting code) so that it generates a different project.qgz but omits the regenerated committed artifact, this setting suppresses the only failing check: the earlier validate, verify, and render steps all inspect the newly generated worktree file rather than the committed version. Such a PR can therefore merge a project.qgz that its own pipeline no longer produces, leaving the scheduled run to report the regression only after it reaches the target branch. Limit the advisory behavior to mismatches proven to come solely from live-source drift rather than ignoring every PR mismatch.
Useful? React with 👍 / 👎.
1 participant
What
The worked-example job regenerates
examples/tartu-development/from three live Estonian services, then diffs the regeneratedproject.qgzagainst the committed one. That last step asks "has the committed example stopped describing its sources?" — a question about the world, not about a branch. Its legend embeds facility counts (Verified municipal schools (n=26)), so one kindergarten opening in Tartu rewrites the file and blocks an unrelated merge.This scopes that one assertion by event: a hard failure on
schedule/workflow_dispatch, advisory onpull_request. Nothing else changes —validate,verify, and the real-QGIS render step all still gate PRs, and they are what actually answer whether a change broke the example.Why now
On 2026-09-09 PR #25, which touches only the CLI, went red here. The evidence, from the job's own uploaded run record and validation report:
data/derived/education_pois.json;project.qgzdiffered at an identical 3698 bytes — consistent with a same-length digit substitution in a legend count.The control: the same step already failed on
mainat d4e1bd3 in the scheduled run of 2026-09-07, two days before that branch existed. The drift preceded the branch.I checked the alternative explanation and it does not hold: extracting the committed
project.qgsand re-zipping it with the pipeline's exact recipe reproduces the committed bytes hash-for-hash, so the zip container is deterministic and this was not a compression-encoder change.This is the workflow's own stated intent — "an outage upstream must not redden unrelated work" — applied to data drift as well as to outages.
Also
A note in
docs/maintainers/debugging.mdon telling the two failure kinds apart from the uploadedworked-example-runartifact, since working that out took downloading CI artifacts and diffing digests.Not fixed here
mainstays red on the schedule until someone with access to the live sources rerunsexamples/tartu-development/pipeline.pyand commits the regeneratedproject.qgzand run record. That is the alarm working correctly.🤖 Generated with Claude Code
https://claude.ai/code/session_01DqoLGyKY5opHGjNMSerpHg