skills: read FNALbuild's table before verifying anything yourself - #15
Open
oksuzian wants to merge 1 commit into
Open
skills: read FNALbuild's table before verifying anything yourself#15oksuzian wants to merge 1 commit into
oksuzian wants to merge 1 commit into
Conversation
mu2e/buildtest runs ~20 checks, most with no separate commit-status context. Reading the statuses API alone reports that a check does not exist when it ran and passed -- g4surfaceCheck and rootOverlaps are the cases that matter, since they cover geometry overlaps across the whole geometry. Adds a "What buildtest already covers" section listing what runs inside buildtest, and an Evidence Rules bullet telling the reviewer to find out what CI verified before verifying it themselves. Both say plainly not to hand-roll a substitute and not to ask an author to run a check CI ran. Prompted by a review that asked the author to run surfaceCheck.fcl when CI had already run it green, backed by a home-made scan that silently covered one of the four solid families the geometry loads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
|
Can't the AI PR system pull previous comments? Why is the CI results table hidden from it? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a note telling the review skill not to verify things CI already verifies.
Why
mu2e/buildtestis not one test — it is about twenty, and most have noseparate commit-status context. The statuses API returns roughly eleven
contexts; the rest appear only as rows in FNALbuild's result table. Reading
statuses alone reports that a check does not exist when it ran and passed.
Two of the invisible ones matter a lot for geometry PRs:
g4surfaceCheck— runsOffline/Mu2eG4/fcl/surfaceCheck.fcl, the Geant4overlap check over the whole geometry. It is graded on log content rather
than exit code (overlaps are G4 warnings, so the job would otherwise exit 0),
with a
LEGAL > 0guard against a vacuous pass.rootOverlaps— an independent overlap check via ROOT.What happened
Reviewing Offline#1933, a building-geometry vertex move, the skill concluded
from the statuses API that CI does not check overlaps. It then wrote its own
Python analysis of the hall geometry and asked the author to run
geom_SurfaceCheck.txtby hand. Bothg4surfaceCheckandrootOverlapswerealready green at that head, so the request was redundant work for the author.
The home-made scan was also wrong in coverage while reading as verified:
Mu2eHallMakerloads four families of solids, and the scan matched onlybuilding.*prefixes — silently dropping 100 dirt, 13 rotated and 3 trapvolumes. It still returned "no overlaps", which is indistinguishable from a
real result. The review shipped a 🟢 claiming "every active hall volume".
Changes
section under Triggering a CI Build: lists what runs inside buildtest, says
not to re-derive it and never to ask an author to run one of those checks,
and says that an independent check is a cross-check of CI, never the
evidence. Points at
Mu2e/codetoolsfor the job lists andMu2e/CIfor thetest vocabulary.
verifying anything yourself.
Net +42 lines, no other content touched. This generalises the rule the skill
already had for the whitespace check, which runs inside buildtest for the same
reason.
🤖 Generated with Claude Code