Skip to content

fix(deep-scan): recover complete coverage without hiding incomplete worker results - #657

Open
mldangelo-oai wants to merge 6 commits into
mainfrom
dev/codex/recover-deep-scan-completeness
Open

fix(deep-scan): recover complete coverage without hiding incomplete worker results#657
mldangelo-oai wants to merge 6 commits into
mainfrom
dev/codex/recover-deep-scan-completeness

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Deep Scans can finish with partial coverage even when every surface has been reviewed. Recover complete coverage only when the saved parent and worker evidence supports it, while retaining findings from unverified replacement drafts.

Changes

  • Apply coverage recovery in the canonical plugin source under plugins/codex-security, using the existing saved-parent reader and generated bundle workflow.
  • Keep missing successful-worker results, incomplete reducer results, malformed completion markers, invalid coverage, and unknown warnings from being promoted to complete. Check required reducer outputs even when a parent supersedes them, and retain warnings across retries.
  • Preserve checkpoint findings when the current worker or parent has an invalid completion marker.
  • Allow lossless duplicate handling to recover coverage regardless of finding order, without changing existing warning text.
  • Exercise recovery with native Python tests over real saved artifacts, finalization, and retries. Bump the bundled plugin to 0.1.84 and verify cache upgrades from 0.1.79 and 0.1.83.

Testing

  • Full Python suite: 1,125 passed, 5 skipped, and 104 subtests passed.
  • Full SDK suite at seed 12345: 2,073 passed, 39 skipped, zero failures.
  • MCP integration suite: all 23 test files passed.
  • All 58 coverage-recovery cases passed. Fourteen cases reproduced incomplete reducer promotion, lost checkpoint findings, or order-dependent duplicate recovery before the fixes.
  • Real bundled Codex cache upgrades passed, including installed helper bytes, preserved credentials/configuration, and SDK-owned artifact restoration.
  • Types and generated-model freshness, Prettier, Ruff 0.16.1, portable source compatibility, generated-source checks, package build, full installed-package smoke, and git diff --check passed.
  • macOS setup: Python used the normal system temporary directory; Bun used a canonical temporary directory. Two Codex transport-fixture failures with the default path alias reproduced on unchanged main and passed with canonical paths.
  • Native Windows/Linux execution is left to CI. No live model scan was run.

Risk and rollout

Recovery remains limited to deep repository coverage with reviewed surfaces and no deferred work. During recovery, unverified sources and unknown warnings preserve partial coverage. Existing handling of omitted completion markers is unchanged. No public CLI flags, arguments, defaults, configuration, or database migrations change. The plugin version bump delivers the corrected helpers to existing cached installations.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

Historical commit contact metadata and automated restricted settings links remain in the existing PR history, so the second attestation stays unchecked. New fixtures are synthetic and new commits use GitHub noreply metadata.

@github-actions github-actions Bot added the bug Something isn't working label Aug 26, 2026
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 26, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-30T04:50:29.585666Z a622ef9 Manual request
🔒 Security Review Completed 2026-08-30T04:52:43.129753Z a622ef9 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: c1d997a12f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2db25481cd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +589 to +592
if (relative != "parent" or not parent_manifest) and not superseded:
source_coverage = draft["coverage"]
source_completeness = source_coverage.get("completeness")
source_complete = draft.get("complete", True) is True

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject malformed parent completion markers before promotion

When the canonical parent manifest contains scan.complete: null, 0, or "false", complete_scan_locked does not reject it because it checks only is False, and this condition exempts that parent from source_complete validation. A Deep Scan with otherwise valid partial coverage, reviewed surfaces, and no deferred rows is therefore promoted to complete, despite an explicit malformed completion marker indicating that the parent result is unverified. Validate the canonical parent's marker here as well so malformed parent drafts remain partial rather than producing a successful scan conclusion.

AGENTS.md reference: sdk/typescript/AGENTS.md:L22-L24

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd66656cad

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

and coverage.get("completeness") in {"complete", "unknown"}
):
coverage["completeness"] = "partial"
if (relative != "parent" or not parent_manifest) and not superseded:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve incomplete reducer results before promotion

For a successful dedup worker whose result exists but has complete: false or partial coverage, worker_id is None; once the canonical parent has complete: true, the preceding superseded predicate is true, so this verification block never records the unverified-coverage warning. If that parent has the motivating partial coverage with reviewed surfaces and no deferred rows, _recover_unsealed_coverage then promotes it to complete and the CLI reports success despite the reducer explicitly remaining incomplete. Inspect reducer completeness before applying parent supersession.

AGENTS.md reference: sdk/typescript/AGENTS.md:L22-L23

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e19ff83712

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1045 to +1048
or re.fullmatch(
r"Recovered finding [0-9]+: "
r"(?:normalized [a-z, ]+|retained stronger duplicate logical finding)\.",
warning,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow lossless duplicate warnings during coverage recovery

When an otherwise complete Deep Scan contains duplicate logical findings with the strongest or equal record first, _recover_unsealed_findings safely retains that record but emits Skipped malformed finding N: duplicate logical finding.; this allowlist accepts only the inverse-order retained stronger duplicate warning. Consequently, equivalent findings seal with partial versus complete coverage solely based on their ordering. Treat the lossless duplicate-discard warning as non-coverage-affecting as well.

AGENTS.md reference: sdk/typescript/AGENTS.md:L23-L23

Useful? React with 👍 / 👎.

Comment on lines +594 to +597
if (relative != "parent" or not parent_manifest) and not superseded:
source_coverage = draft["coverage"]
source_completeness = source_coverage.get("completeness")
source_complete = draft.get("complete", True) is True

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Require verified results before superseding checkpoints

When a succeeded discovery worker's current result has a malformed completion marker such as null, 0, or "false", this strict check records a warning only after the earlier supersession predicate has accepted that result via current.get("complete") is not False. A non-stopped merge then skips the worker's older checkpoints entirely, so any valid findings present only in those checkpoints disappear from the sealed partial report. Require complete is True before superseding checkpoint history, or continue merging the older findings.

AGENTS.md reference: sdk/typescript/AGENTS.md:L23-L23

Useful? React with 👍 / 👎.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Current head: a622ef9af66758cef111812c5382dec4751a782b.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex security review

Current head: a622ef9af66758cef111812c5382dec4751a782b.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: a622ef9af6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant