feat(ci): agent gate for knowledge PRs — transferability, duplication, fact check - #155
Merged
Conversation
…, fact check The mechanical wiki gates in test.yml cover format and countable duplicates; the semantic checks (transferability, semantic duplication, fact-checking of cited sources) existed only as self-reported steps inside the knowledge-flush session. This adds an independent CI gate that runs Claude Code headless (anthropics/claude-code-action@v1, subscription OAuth token) over the PR diff on every PR touching wiki/**. - pull_request_target with two guards: a job-level trust gate (base-repo or maintainer-fork PRs only) and a base-ref-only checkout — the untrusted PR head is read exclusively via gh pr diff. - Verdict via --json-schema structured output; the gate fails closed when the agent produces no verdict, and fails on any blocker finding. - Findings are posted back to the PR as a comment on failure. - tests/wiki-agent-gate.bats pins both security guards, the fail-closed step, and the schema/prompt contract, each with a negative control. Requires the CLAUDE_CODE_OAUTH_TOKEN repo secret (claude setup-token). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RJsPDQCz3wNt2qk4opqcKG
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an agent-run CI gate for the three semantic checks scripts cannot do, closing the gap where transferability, semantic duplication, and fact-checking existed only as self-reported steps inside the knowledge-flush session:
.github/workflows/wiki-agent-gate.yml— on every PR touchingwiki/**, runs Claude Code headless viaanthropics/claude-code-action@v1(subscription OAuth token, no per-token billing). Verdict arrives as--json-schemastructured output; the gate fails closed when the agent produces no verdict and fails on anyblockerfinding. Findings are posted back to the PR as a comment on failure..github/wiki-agent-gate-prompt.md— the check definitions: transferability (org-internal directives are blockers), semantic duplication (Grep the existing wiki before accepting a new page/row), fact check (WebFetch each cited source and confirm the quote supports the directive; unfetchable → advisory, never guessed).tests/wiki-agent-gate.bats— 12 doc-gate tests pinning the security guards, the fail-closed step, and the schema/prompt contract, each positive check paired with a negative control.Security (pull_request_target)
Knowledge PRs come from a fork, and plain
pull_requestwithholds secrets there — so this usespull_request_targetwith the two guards the action's security docs prescribe:choiyounggi,dch0202-rsquare) start the job.gh pr diff, and the prompt file instructs it to treat instruction-shaped text inside the diff as a finding, not as instructions.Setup required before this gate can run
The repo secret
CLAUDE_CODE_OAUTH_TOKENmust be added once: runclaude setup-tokenlocally, thengh secret set CLAUDE_CODE_OAUTH_TOKEN.Verification
bats tests/wiki-agent-gate.bats→ 12/12 passwiki/**changes: directive count and structure gates unaffected🤖 Generated with Claude Code
https://claude.ai/code/session_01RJsPDQCz3wNt2qk4opqcKG