Skip to content

ci: run every job when CI plumbing changes - #1286

Open
ogenstad wants to merge 1 commit into
stablefrom
pog-em/run-jobs-on-ci-change-jr3uf
Open

ci: run every job when CI plumbing changes#1286
ogenstad wants to merge 1 commit into
stablefrom
pog-em/run-jobs-on-ci-change-jr3uf

Conversation

@ogenstad

@ogenstad ogenstad commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Why

Path filtering in files-changed decides which jobs run. Today a change to the CI plumbing itself does not turn on every job, so a bad edit to a workflow can land green and then break the next unrelated PR - the job that would have caught it was skipped on the PR that introduced it.

Three concrete gaps, confirmed by simulating dorny/paths-filter with picomatch against the current filters:

Changed file Jobs that silently skipped
.github/workflows/ci.yml documentation
.github/file-filters.yml markdown-lint, action-lint, uv-lock-check, documentation
.github/workflows/define-versions.yml python-lint, unit-tests, integration-tests, validate-generated-documentation, documentation

The third is the worst of them: define-versions.yml pins the UV_VERSION that every Python job consumes, and a bad pin there was never exercised by unit or integration tests.

This is observable in history. #1192, a ci.yml-only PR, shows documentation skipped in run 29730072415.

Goal: any change to the CI plumbing runs the full pipeline.

Non-goals: no change to job selectivity for ordinary code, docs, or dependency PRs. Scoped to workflows and the filter file, not all of .github/** - dependabot.yml, labeler.yml, CODEOWNERS, and issue templates do not affect whether a job runs correctly, and including them would mean a full pipeline on every label tweak.

What changed

Behavioral changes:

  • Any change under .github/workflows/** or to .github/file-filters.yml now runs every CI job, so a broken job definition fails on the PR that introduces it.
  • Selectivity is otherwise untouched: python-only, docs-only, and generated-docs-only PRs run exactly the same job set as before.

Implementation notes:

  • .github/file-filters.yml: widened ci_config to .github/workflows/** plus .github/file-filters.yml, pointed github_workflows at it, and folded it into documentation_all and documentation_generated_all. Every output that gates a job now includes *ci_config.
  • .github/workflows/ci.yml: parenthesized the validate-generated-documentation condition. It read ... && A || B, and && binds tighter than ||, so documentation_generated == 'true' alone satisfied the whole expression and bypassed the always() && !cancelled() && !contains(needs.*.result, 'failure') guards. Latent before this PR; the filter change makes that output true on every CI-config change, which would have started firing it.

What stayed the same: no job definitions, runners, timeouts, or needs edges changed. The only ci.yml edit is the parenthesization above.

How to review

Two files, 8 insertions. The load-bearing question is whether the widened ci_config over-triggers - see the simulation table below for the answer.

Worth extra scrutiny: github_workflows is now an alias to ci_config, so action-lint also fires on .github/file-filters.yml changes. That is intentional (actionlint lints the whole workflow directory regardless of what changed) but it does widen that job's trigger.

How to test

This PR is its own test case: it changes both ci.yml and file-filters.yml, so every job in the pipeline should run on it. Previously documentation would have been skipped.

Static checks:

uv run yamllint -s .
bash <(curl -sL https://raw.githubusercontent.com/rhysd/actionlint/main/scripts/download-actionlint.bash) && ./actionlint -color

Both pass. Filter behavior was verified by flattening the YAML anchors and matching sample changed-file sets with picomatch using {dot: true}, the same options dorny/paths-filter uses:

Changed file Skipped before Skipped after
.github/workflows/ci.yml documentation none
.github/file-filters.yml markdown-lint, action-lint, uv-lock-check, documentation none
.github/workflows/define-versions.yml python-lint, documentation, validate-generated-documentation, unit-tests, integration-tests none
infrahub_sdk/client.py yaml-lint, markdown-lint, action-lint, uv-lock-check, documentation unchanged
docs/docs/guides/foo.mdx 7 jobs unchanged
docs/docs/infrahubctl/branch.mdx 6 jobs unchanged

Impact & rollout

  • Backward compatibility: no behavior change outside CI.
  • Performance: CI-config PRs now also run the documentation job (roughly 5 minutes, ubuntu-22.04), and define-versions.yml PRs now run unit and integration tests. That is the point of the change: CI-config PRs are rare, and the alternative is discovering the breakage on someone else's PR.
  • Config/env changes: none.
  • Deployment notes: safe to merge.

Checklist

  • Tests added/updated - N/A, CI configuration only. Filter behavior verified by simulation (table above).
  • Changelog entry added - N/A, no user-facing change (ci/skip-changelog).
  • External docs updated - N/A, no user-facing or ops-facing change.
  • Internal .md docs updated - N/A, AGENTS.md documents local commands, not CI path filtering.

Follow-up, not in this PR

ci.yml exports helm: ${{ steps.changes.outputs.helm_all }}, but no helm_all filter exists and nothing consumes the output, so it is always empty. Harmless today, but it is the kind of line someone later gates a job on. Deliberately left out of scope.


Summary by cubic

Makes any change to the CI plumbing run the full pipeline. Previously, editing a workflow or the path-filter file could silently skip jobs — for example, a define-versions.yml-only change skipped unit and integration tests, letting a bad UV_VERSION pin land and break the next unrelated PR.

  • Widened ci_config from ci.yml alone to all of .github/workflows/** plus .github/file-filters.yml, and folded it into every output that gates a job.
  • Pointed github_workflows at ci_config, so action-lint also fires when the filter file changes.
  • Parenthesized the validate-generated-documentation condition, which previously read ... && A || B and let the docs check bypass the always() && !cancelled() && !contains(...) guards.

Written for commit f1552a8. Summary will update on new commits.

Review in cubic

A change to a workflow file or to the path filters themselves could
previously leave jobs skipped, so a bad edit landed green and broke the
next unrelated PR instead.

Three concrete gaps, confirmed by simulating dorny/paths-filter against
the old filters:

- ci.yml only: `documentation` skipped, because `documentation_all` never
  included `ci_config`.
- file-filters.yml only: `markdown-lint`, `action-lint`, `uv-lock-check`
  and `documentation` skipped, because `github_workflows` was
  `.github/workflows/*.yml`, which does not match a file one level up.
- define-versions.yml only: `python-lint`, `unit-tests`,
  `integration-tests` and `validate-generated-documentation` skipped,
  because `ci_config` listed only `ci.yml` - so a bad `UV_VERSION` pin,
  consumed by every Python job, went untested.

Widen `ci_config` to all of `.github/workflows/**` plus
`.github/file-filters.yml`, point `github_workflows` at it, and fold it
into `documentation_all` and `documentation_generated_all`. Every output
that gates a job now includes it.

Also parenthesize the `validate-generated-documentation` condition. It
read `... && A || B`, and `&&` binds tighter than `||`, so
`documentation_generated == 'true'` alone satisfied the whole expression
and bypassed the `always() && !cancelled() && !contains(needs.*.result,
'failure')` guards. Latent before; the change above makes that output
true on every CI-config change, which would have started firing it.

Verified with `yamllint -s .` and `actionlint`, plus a picomatch
simulation showing zero skipped jobs for all three CI-config cases and
no change in selectivity for python-only, docs-only or generated-docs-only
changes.
@ogenstad ogenstad added group/ci Issue related to the CI pipeline type/bug Something isn't working as expected ci/skip-changelog Don't include this PR in the changelog labels Aug 26, 2026
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying infrahub-sdk-python with  Cloudflare Pages  Cloudflare Pages

Latest commit: f1552a8
Status: ✅  Deploy successful!
Preview URL: https://ac237a37.infrahub-sdk-python.pages.dev
Branch Preview URL: https://pog-em-run-jobs-on-ci-change.infrahub-sdk-python.pages.dev

View logs

@codecov

codecov Bot commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@           Coverage Diff           @@
##           stable    #1286   +/-   ##
=======================================
  Coverage   84.16%   84.16%           
=======================================
  Files         147      147           
  Lines       13045    13045           
  Branches     1930     1930           
=======================================
  Hits        10979    10979           
  Misses       1503     1503           
  Partials      563      563           
Flag Coverage Δ
integration-tests 39.01% <ø> (+<0.01%) ⬆️
python-3.10 56.97% <ø> (-0.02%) ⬇️
python-3.11 56.99% <ø> (ø)
python-3.12 56.97% <ø> (-0.02%) ⬇️
python-3.13 56.99% <ø> (ø)
python-3.14 56.97% <ø> (-0.02%) ⬇️
python-filler-3.12 23.68% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Re-trigger cubic

@ogenstad
ogenstad marked this pull request as ready for review August 26, 2026 18:57
@ogenstad
ogenstad requested a review from fatih-acar August 26, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/skip-changelog Don't include this PR in the changelog group/ci Issue related to the CI pipeline type/bug Something isn't working as expected

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant