From 25fad4962ec1d5e31f2d400eca15be609da1f86e Mon Sep 17 00:00:00 2001
From: Paul Keen <125715+pftg@users.noreply.github.com>
Date: Tue, 25 Aug 2026 21:18:37 +0200
Subject: [PATCH] ci: run the full matrix automatically on the paths that break
 master (#281)

The matrix stays off PRs for free-tier minutes, and the consequence is that the
job which breaks master is a job that never ran on the PR. Three breakages this
week, all invisible on a green PR for exactly that reason:

- a Rails 7.1-only constant (#283)
- JRuby not implementing Kernel#fork (#283)
- a zero-width skip_area mask (#280)

Every one was found by adding `full-ci` BY HAND after master had already gone
red, which is a process that works only when someone remembers.

All three came from `test/`, `gemfiles/` or `.github/`, so the matrix now runs
automatically when a PR touches those. The label stays for everything else.

Paths are asked for over the API rather than `git diff`: checkout is depth-1, so
the base commit is not in the clone to diff against.

`lib/` is deliberately NOT on the list. It changes on nearly every PR, and the
functional and minimal-setup jobs already cover it -- putting it here would run
25 cells on almost everything and give back the cost decision the exclusion
exists to make. That is a trade, not a claim that `lib/` is safe, and
CONTRIBUTING.md says so.

CONTRIBUTING.md also gets the two reading rules that cost real time this week:
`cancelled` is not a pass, and `gh run list --branch master` without
`--workflow Test` will hand you whichever workflow ran last -- it reported a
Dependabot success while `Test` was failing on the same commit.

Detection self-tested against real paths; YAML validated.
---
 .github/workflows/test.yml | 41 ++++++++++++++++++++++++++++++++++----
 CONTRIBUTING.md            | 32 +++++++++++++++++++++++++++++
 2 files changed, 69 insertions(+), 4 deletions(-)

diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 441ec8cf..0e5ecf04 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -68,7 +68,31 @@ jobs:
       contents: read
       pull-requests: write
 
+    outputs:
+      # Whether this PR touches a path that has historically broken master.
+      high_risk_paths: ${{ steps.risk.outputs.high }}
+
     steps:
+      - name: Detect paths that have historically broken master
+        id: risk
+        if: github.event_name == 'pull_request'
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          # Asked over the API, not `git diff`: checkout is depth-1, so the base
+          # commit is not in the clone to diff against.
+          files=$(gh api \
+            "repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files" \
+            --paginate --jq '.[].filename')
+          if grep -qE '^(test/|gemfiles/|\.github/)' <<<"$files"; then
+            echo "high=true" >> "$GITHUB_OUTPUT"
+            echo "High-risk paths touched -- the full matrix will run on this PR:"
+            grep -E '^(test/|gemfiles/|\.github/)' <<<"$files" | sed 's/^/  /'
+          else
+            echo "high=false" >> "$GITHUB_OUTPUT"
+            echo "No high-risk paths touched; the full matrix stays off this PR."
+          fi
+
       - name: Checkout code
         uses: actions/checkout@v7
 
@@ -98,14 +122,23 @@ jobs:
 
   matrix:
     name: Test Ruby & Rails
-    # Cost-intentional: full matrix stays off PRs by default (free-tier
-    # Actions minutes). Runs on master pushes, manual dispatch, the weekly
-    # scheduled drift check, and PRs opted in via the 'full-ci' label.
+    # Cost-intentional: the full matrix stays off PRs that cannot plausibly
+    # break it (free-tier Actions minutes). It runs on master pushes, manual
+    # dispatch, the weekly drift check, PRs opted in via the 'full-ci' label,
+    # and -- since #281 -- automatically on any PR touching `test/`,
+    # `gemfiles/` or `.github/`.
+    #
+    # Those three paths are not a guess. Every master breakage traced this week
+    # came from one of them, and each was invisible on a green PR precisely
+    # because the matrix had not run: a Rails 7.1-only constant (#283), JRuby
+    # lacking Kernel#fork (#283), and a zero-width skip_area mask (#280). All
+    # three were found only by adding the label BY HAND after the fact.
     if: >
       github.ref == 'refs/heads/master' ||
       github.event_name == 'workflow_dispatch' ||
       github.event_name == 'schedule' ||
-      contains(github.event.pull_request.labels.*.name, 'full-ci')
+      contains(github.event.pull_request.labels.*.name, 'full-ci') ||
+      needs.functional-test.outputs.high_risk_paths == 'true'
     needs: [ functional-test ]
     runs-on: ubuntu-latest
     # Must fit `max_attempts * timeout_minutes` below, plus ~1 min of setup,
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index dc658049..b195423c 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -68,6 +68,38 @@ bundle exec standardrb
 bundle exec standardrb --fix
 ```
 
+## CI: a green PR does not always mean a green master
+
+**The full `Test Ruby & Rails` matrix does not run on every PR.** It is off by
+default to stay inside free-tier Actions minutes, which means the job that breaks
+`master` can be a job that never ran on your PR.
+
+It runs automatically when:
+
+- the PR touches **`test/`, `gemfiles/` or `.github/`** — the three paths every
+  recent master breakage came from;
+- you add the **`full-ci`** label;
+- or the push is to `master`, a manual dispatch, or the weekly drift check.
+
+**Add `full-ci` by hand** if your PR could behave differently across Ruby or
+Rails versions and does not touch those paths — anything version-conditional
+(`defined?`, `respond_to?`, `RUBY_VERSION`), anything touching subprocess or
+environment handling, or anything you would be surprised to see break on JRuby.
+`lib/` is deliberately **not** on the automatic list: it changes on nearly every
+PR, and the functional and minimal-setup jobs already cover it. That trade is a
+cost decision, not a claim that `lib/` is safe.
+
+Two things about reading CI results here:
+
+- **`cancelled` is not a pass.** It means a later push superseded the run, or
+  fail-fast killed the cell before it reported. It occupies the same slot as a
+  verdict while carrying none — a JRuby lane sat broken for 15 consecutive runs
+  looking exactly like this.
+- **After merging, check `master`.** `gh run list --branch master --workflow Test
+  --limit 1`. A red `master` blocks the next merge. Pass `--workflow Test`: without
+  it you get whichever workflow ran last, which has already reported a Dependabot
+  success while `Test` was failing on the same commit.
+
 ## Coding Conventions
 
 ### Style