feat(verify-pr): prefetch CI check-run outcomes into sandbox input

[mrizzi]

Summary

In the CI-dispatched (sandbox) verify-pr path the CI Status check had no data to work from — the sandbox has no gh CLI and no network egress — so it degraded to WARN "CI status could not be retrieved in sandbox mode" (observed on PR #301 during the TC-6192 E2E acceptance run). This defeats the TC-6180 premise that verify-pr consumes the CI result as input data.

This extends the existing split-trust prefetch: the pre_script (on the trusted runner, where GH_TOKEN and the head SHA live) fetches the PR head-SHA CI check-run outcomes and embeds them in the sandbox input bundle. correctness.md Check 1 reads them in sandbox mode; interactive mode keeps fetching via gh on demand.

Changes

• pre-verify-pr.sh — prefetch repos/<repo>/commits/<COMMIT_SHA>/check-runs into check-runs.json, reduced to name/status/conclusion/details_url.
• pre_verify_pr.py — build_github_bundle embeds github.check_runs (defaults to []); _github_from_dir reads check-runs.json.
• verify-pr-input.schema.json — add required github.check_runs. The github object is additionalProperties:false, so the new key must be declared or Step 0.7 validation rejects the bundle. (Out of the task's Files-to-Modify list but required for the feature; approved.)
• correctness.md — Check 1 sandbox-mode branch reads pre-fetched check-runs and maps conclusions to PASS/WARN/FAIL; 1b documents that failure logs are unavailable in the sandbox (analysis derived from diff + check name/conclusion, details_url carried into the sub-task).
• dispatch-template.md, SKILL.md Step 0.7 — document the github.check_runs key.
• test_pre_verify_pr.py — cover check_runs in the bundle, the transform, the CLI dir path, and the empty-default.

Data passed from CI

Only outcome metadata per check-run — name, status, conclusion, details_url — not the failure logs (kept out to keep the bundle small; the sandbox cannot fetch them anyway).

Verification

• python3 -m pytest plugins/sdlc-workflow/scripts/ -q → 144 passed
• uvx skillsaw → 0 errors
• claude plugin validate plugins/sdlc-workflow → passed

Deferred: E2E re-verification on a live dispatched PR (CI Status = PASS on a green PR, FAIL on a red PR, both sourced from prefetched data) and the coupled fullsend.md acceptance note — both await a live CI dispatch run.

Implements TC-6257

🤖 Generated with Claude Code

Summary by Sourcery

Enable sandbox verify-pr to evaluate CI status and investigate failures using trusted-runner-prefetched GitHub data.

New Features:

• Provide sandbox verify-pr runs with prefetched GitHub CI check-run outcomes and failed-check logs from the trusted runner.

Bug Fixes:

• Enable CI Status verification in network-isolated sandbox mode instead of falling back to an unavailable-data warning.

Enhancements:

• Extend the verify-pr input bundle and schema with CI outcomes and an on-demand path for failure logs while preserving interactive-mode fetching.
• Update correctness guidance and dispatch documentation to consume prefetched CI data and logs.
• Mount failed-check logs separately so large log contents are read only when a CI failure requires analysis.

Deployment:

• Re-pin and lock the verify-pr harness to deliver the updated prefetch and sandbox behavior.

Documentation:

• Document prefetched CI outcomes and failure-log handling in the verify-pr skill, dispatch template, and fullsend workflow notes.

Tests:

• Add coverage for CI check-run bundling, defaults, directory transformation, and conditional failure-log path handling.

[sourcery-ai]

Reviewer's Guide

The PR extends the trusted-runner prefetch pipeline with head-SHA CI check-run metadata, makes sandbox CI verification evaluate that embedded data without GitHub access, retains on-demand gh behavior interactively, and updates the schema, documentation, and tests for the new required bundle field.

Sequence diagram for prefetched CI status in sandbox verification

sequenceDiagram
    participant Runner as Trusted runner
    participant GitHub as GitHub API
    participant Bundle as Sandbox input bundle
    participant Sandbox as Sandbox verifier
    participant Correctness as Correctness Check 1

    Runner->>GitHub: check-runs for COMMIT_SHA
    GitHub-->>Runner: name, status, conclusion, details_url
    Runner->>Bundle: build_github_bundle(check_runs)
    Bundle->>Sandbox: github.check_runs
    Sandbox->>Correctness: CI Status input
    Correctness->>Correctness: Map conclusions to PASS, WARN, or FAIL
    Correctness-->>Sandbox: CI verdict

Loading

File-Level Changes

Change	Details	Files
Prefetch head-SHA CI check-run metadata on the trusted runner and include it in the sandbox GitHub input bundle.	Fetch paginated check runs and flatten the endpoint’s per-page response. Retain only name, status, conclusion, and details URL. Load the new JSON file into the bundle and default missing data to an empty list.	plugins/sdlc-workflow/scripts/pre-verify-pr.sh plugins/sdlc-workflow/scripts/pre_verify_pr.py
Make sandbox CI Status verification consume prefetched outcomes while preserving interactive-mode fetching.	Map check-run status and conclusions to pass, pending, or failed outcomes. Treat an empty check-run list as no checks configured and passing. Skip sandbox log retrieval and use metadata, diff, and details URLs for failure analysis and follow-up tasks.	plugins/sdlc-workflow/skills/verify-pr/correctness.md plugins/sdlc-workflow/skills/verify-pr/SKILL.md plugins/sdlc-workflow/skills/verify-pr/dispatch-template.md
Require and test the new check-run field throughout the verify-pr input contract.	Declare required github.check_runs in the closed GitHub schema. Cover bundle construction, empty defaults, transformation, and directory-based CLI loading.	plugins/sdlc-workflow/schemas/verify-pr-input.schema.json plugins/sdlc-workflow/scripts/test_pre_verify_pr.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 3 issues

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="plugins/sdlc-workflow/scripts/pre-verify-pr.sh" line_range="191" />
<code_context>
+# before dispatch. Reduced to the fields the verdict needs (name/status/conclusion
+# and details_url for the failure-log link) to keep the bundle small. A PR with no
+# checks yields an empty array — consistent with the reviews/comments empties.
+gh api --paginate --slurp "repos/${PR_REPO}/commits/${COMMIT_SHA}/check-runs" \
+  | jq '[.[].check_runs[] | {name, status, conclusion, details_url}]' > "${PRE_OUTPUT_DIR}/check-runs.json"

 echo "GitHub read bundle prefetched to ${PRE_OUTPUT_DIR}"

</code_context>
<issue_to_address>
**issue (bug_risk):** The prefetch includes every check run returned for the commit, including superseded failed runs from workflow reruns, and Check 1 treats any failed entry as a CI failure. A commit with a failed first run followed by a successful rerun is therefore reported as FAIL even though the current check result is passing.

**Triggers:** When a workflow or check is rerun for the same head SHA after an earlier failure.

**Suggested fix:** Select only the current/latest check run for each check identity, or use an API/result representation that matches the PR's effective check status rather than all historical runs.

```suggestion
  | jq '[.[].check_runs[]] | group_by(.name) | map(max_by(.id) | {name, status, conclusion, details_url})' > "${PRE_OUTPUT_DIR}/check-runs.json"
```
</issue_to_address>

### Comment 2
<location path="plugins/sdlc-workflow/skills/verify-pr/correctness.md" line_range="44-45" />
<code_context>
+pre-fetched check-run outcomes from the CI Status input (each entry has `name`,
+`status`, `conclusion`, `details_url`). Map each entry to a status:
+
+- `conclusion` of `success`/`neutral`/`skipped` → pass
+- `conclusion` of `failure`/`timed_out`/`cancelled`/`action_required` → failed
+- `status` not `completed` (i.e. `queued`/`in_progress`) → pending
+- an empty check-runs list → no checks configured (treat as pass; note it in the
+  evidence)
+
</code_context>
<issue_to_address>
**issue (bug_risk):** The sandbox mapping does not define the GitHub check conclusions `startup_failure` or `stale`, even though both are valid completed check-run conclusions that indicate a check did not pass. Those entries receive no specified verdict and can be omitted from the failed-check path, allowing CI Status to be reported as PASS or otherwise misclassified.

**Triggers:** When a check run completes with `startup_failure` or `stale`.

**Suggested fix:** Explicitly classify all non-success terminal conclusions, including `startup_failure` and `stale`, as failed; treat only the documented pass conclusions as passing.

```suggestion
- `status` of `completed` with `conclusion` of `success`/`neutral`/`skipped` → pass
- `status` of `completed` with any other `conclusion` → failed
```
</issue_to_address>

### Comment 3
<location path="plugins/sdlc-workflow/skills/verify-pr/correctness.md" line_range="66-69" />
<code_context>
-For each failed CI check, fetch the failure logs:
+**Sandbox mode:** the failure logs cannot be fetched — there is no `gh` CLI or
+network egress. Skip the `gh run view`/`gh run list` commands below and derive the
+failure analysis in step 1c from the PR diff plus the failed check's `name`,
+`conclusion`, and `details_url` from the pre-fetched CI Status input. Still emit the
+`create-sub-task` action (step 1d); include the `details_url` so a human can open
+the full log.
+
+**Interactive mode:** for each failed CI check, fetch the failure logs:
</code_context>
<issue_to_address>
**issue:** Sandbox Check 1b says to derive failure analysis without logs, but the unchanged Check 1c instructions still require analyzing the failure log to identify the specific failure and root error. In sandbox mode no log exists, so the sub-agent is given contradictory instructions and cannot satisfy the required evidence format from the prefetched fields alone.

**Triggers:** When a prefetched check is failed in sandbox mode.

**Suggested fix:** Add an explicit sandbox branch to Check 1c that requires only diff-based analysis and the check metadata, and removes the log-specific requirements for that mode.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 3 findings to address first, and if the prefetched check-run data is incomplete or mapped incorrectly, the sandbox could report the wrong CI verdict and create an unnecessary verification sub-task or miss a failed check. Reverting stops the behavior, but any sub-task or verdict produced before the revert would need manual correction.

Blocking findings: plugins/sdlc-workflow/scripts/pre-verify-pr.sh:191, plugins/sdlc-workflow/skills/verify-pr/correctness.md:45, plugins/sdlc-workflow/skills/verify-pr/correctness.md:69

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

[sourcery-ai]

issue (bug_risk): The prefetch includes every check run returned for the commit, including superseded failed runs from workflow reruns, and Check 1 treats any failed entry as a CI failure. A commit with a failed first run followed by a successful rerun is therefore reported as FAIL even though the current check result is passing.

Triggers: When a workflow or check is rerun for the same head SHA after an earlier failure.

Suggested fix: Select only the current/latest check run for each check identity, or use an API/result representation that matches the PR's effective check status rather than all historical runs.

Suggested change

	| jq '[.[].check_runs[] | {name, status, conclusion, details_url}]' > "${PRE_OUTPUT_DIR}/check-runs.json"
	| jq '[.[].check_runs[]] | group_by(.name) | map(max_by(.id) | {name, status, conclusion, details_url})' > "${PRE_OUTPUT_DIR}/check-runs.json"

[sourcery-ai]

issue (bug_risk): The sandbox mapping does not define the GitHub check conclusions startup_failure or stale, even though both are valid completed check-run conclusions that indicate a check did not pass. Those entries receive no specified verdict and can be omitted from the failed-check path, allowing CI Status to be reported as PASS or otherwise misclassified.

Triggers: When a check run completes with startup_failure or stale.

Suggested fix: Explicitly classify all non-success terminal conclusions, including startup_failure and stale, as failed; treat only the documented pass conclusions as passing.

Suggested change

	- `conclusion` of `success`/`neutral`/`skipped` → pass
	- `conclusion` of `failure`/`timed_out`/`cancelled`/`action_required` → failed
	- `status` of `completed` with `conclusion` of `success`/`neutral`/`skipped` → pass
	- `status` of `completed` with any other `conclusion` → failed