Skip to content

fix(proof): fail closed on incomplete harbor evaluate (follow-up #266) - #268

Closed
echobt wants to merge 1 commit into
mainfrom
cursor/harbor-fail-closed-evaluate-402d
Closed

echobt wants to merge 1 commit into
mainfrom
cursor/harbor-fail-closed-evaluate-402d

Conversation

@echobt

@echobt echobt commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #266 (41b51ac3). Greptile P1 fail-closed fixes from b9c47391 landed on cursor/harbor-evaluate-artefact-d8ea after that PR was squash-merged and were never on main. This PR cherry-picks that commit onto current main.

Harbor evaluate now fails closed instead of publishing a partial or escaped score:

  1. Nonzero Harbor exitrun-harbor and summarize.py refuse to write report.json from an incomplete or failed trial set.
  2. Score every measured trial — the 256-row cap is evidence serialization only; primary_value is the mean of the full measured set.
  3. Incomplete inspect scans — hitting the file/byte limit fails off-limits rules rather than treating truncated absence as a clean pass.
  4. resolve_agent origin checkimport_path modules whose origin is outside the staged artefact (stdlib, inherited PYTHONPATH, etc.) are rejected; evaluate PYTHONPATH is the artefact parent only.
  5. Unknown rule IDs — inspect fails closed instead of leaving an unanswered item.

Does not touch defer/stub/DNS bake.

Greptile

Every PR is reviewed by Greptile before merge. Config: .greptile/.

  • Greptile has reviewed this PR; findings are fixed or answered
  • If the bot was silent, I commented @greptileai review

Test plan

  • Harbor adaptor unit tests (deploy/guest/runners/rlm_fc_in_guest_harbor/tests/run.sh) — all passed (test_summarize.py, test_inspect_scan.py, test_resolve_agent.py, test_adaptor.sh)
  • cargo test --workspace not required (guest Python/shell only; no Rust crates)
  • cargo fmt --all -- --check not required (no Rust)

Risk

Guest Harbor adaptor only. Changes scoring fail-closed behavior for the in-guest Harbor runner (nonzero exit, truncated inspect, escaped import path, unknown rules). No BASE_* env, CVM measurement, signature domain, or emission-path changes. No image bake.

Naming

I did not rename BASE_* environment variables, deployed host paths
(/opt/base, /run/base, …), GHCR baseintelligence/base package names, or
base-*-v1 cryptographic domain tags, unless this PR’s purpose is a coordinated
cutover documented in docs/NAMING.md.

Open in Web Open in Cursor 

Reject nonzero Harbor exits, score every measured trial, fail truncated
off-limits scans and unknown inspect rules, and refuse import paths whose
origin is outside the staged artefact.

Co-authored-by: Mathis <echobt@users.noreply.github.com>
@cursor

cursor Bot commented Sep 10, 2026

Copy link
Copy Markdown

@greptileai review

@echobt

echobt commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

Closing as duplicate of #267 (same fail-closed follow-up / same file set +395/-37). Keeping #267 as the sole follow-up for Architecte HOLD → secure bake.

@echobt echobt closed this Sep 10, 2026
@greptile-apps

greptile-apps Bot commented Sep 10, 2026

Copy link
Copy Markdown

Greptile Summary

Summary

An oversized artifact file can hide prohibited off-limits markers from the Harbor inspection. The scanner skips files above its per-file size limit without recording that the scan was incomplete, allowing both off-limits checks and their combined gate to pass. This must be fixed before merging.

Confidence Score: 4/5

Not safe to merge: the Harbor off-limits gate can report a clean result while prohibited markers are present in an oversized artifact file.

The failure was reproduced using the production inspection path with otherwise identical files on opposite sides of the configured size limit.

Files Needing Attention: deploy/guest/runners/rlm_fc_in_guest_harbor/inspect_scan.py

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex produced a finding-proof for a posted P1 finding and prepared four supporting artifacts: an oversized off-limits reproduction, the at-limit inspection output, the oversized inspection output, and the inspect scanner test output.
  • Validated the inspection reproduction by running it at the per-file limit and one byte above, confirmed that only the oversized case omits artifact text, keeps incomplete: false, and reports a passing combined off-limits gate, and ran the existing inspect-scan unit suite with all tests passing.
  • T-Rex produced a second finding-proof for another posted P1 finding.
  • Ran the two-mode oversized-off-limits reproduction and the inspect_scan tests; at 131,072 bytes markers were returned; at 131,073 bytes no artifact text and off_limits_gate_all_rules_pass was true; four tests in the inspect_scan suite passed.

View all artifacts

T-Rex Ran code and verified through T-Rex

Comments Outside Diff (1)

  1. deploy/guest/runners/rlm_fc_in_guest_harbor/inspect_scan.py, line 107-108 (link)

    P1 Oversized Files Skip Gates

    Files larger than MAX_FILE_BYTES are ignored without marking the inspection incomplete. As a result, the off-limits checks receive no content to match and report a pass: a 131,073-byte Python artifact containing prohibited short-circuit and TB4 markers passes both checks and the combined gate. Mark every size-limited skip as incomplete, or fail the affected checks closed, before merging.

    Artifacts

    Oversized off-limits reproduction

    • The reproduction creates boundary-sized artifacts containing every relevant prohibited marker and invokes the inspection path.

    At-limit inspection output

    • The 131,072-byte artifact is inspected, its markers are found, and both off-limits checks fail.

    Oversized inspection output

    • The 131,073-byte artifact is skipped without an incomplete result, so both off-limits checks and their combined gate pass.

    Inspect scanner test output

    • The existing inspect-scan unit suite completed successfully with four passing tests.

    View artifacts

    T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "fix(proof): fail closed on incomplete ha..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants