Skip to content

docs: wave 2 field test on 3.16.1 - developer report + three run reports - #72

Open
devin-ai-integration[bot] wants to merge 2 commits into
field-gatesfrom
devin/1787924846-wave2-field-report
Open

docs: wave 2 field test on 3.16.1 - developer report + three run reports#72
devin-ai-integration[bot] wants to merge 2 commits into
field-gatesfrom
devin/1787924846-wave2-field-report

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Why

Wave 1 (#70) covered about half the method and found three surfaces that volunteered false statements to
a sponsor. You patched those in #71. Wave 2 was built to close the coverage gap and to attack the fixes
rather than confirm them: three FDEs, three engagements on three real client repos, entered cold from the
README, assigned so that all 37 references, every CLI verb and every adapter had a named owner - including
the 15 references nobody had touched (the whole commercial/exec path) and the never-driven LOCAL-LLM
adapter.

Based on field-gates (3.16.1 @ a1d1ec1), not Main, because that is the revision under test and the
only one where npm run check is green.

What changed

Docs only - four field reports, no source touched:

  • docs/field-reports/2026-08-28-wave2-developer-report.md - the consolidated report: verdict, the
    fixed-the-instance-not-the-class analysis of 3.16.1, defects ranked blocker → papercut with exact
    command output, 37-reference coverage with per-artifact send/edit/never verdicts, and eight ordered
    changes.
  • three per-run reports (exec-commercial-health, deep-engineering-localllm, lifecycle-redteam).

Headline, so you can judge whether the rest is worth reading:

  • @fde routed to the right reference unprompted in 48/51 exercised cases, and the untested
    business-case → board-memo → red-team chain works end to end - it found two real gaps in an FDE's own
    board memo.
  • Zero data-loss defects under a deliberate attack (racing writes, 5MB files, symlinks, CRLF, read-only
    mounts, missing git identity), and the <private> sentinel appeared in 0 of 12 output surfaces.
  • Your three fixes each hold for the reported case and leak in the same class. A bullet naming nobody
    still mints a stakeholder (batch, still, Friday, Dr, Zo); two people sharing a first name
    collapse so one Will's green clears the other's amber; .last-write keeps the secret after
    redact --apply; an empty reality.md warns nothing.
  • Four new blockers: receipts asserts "nothing was ever logged" over an entry sealed by an unclosed
    <private> (same class as the wave-1 bugs, and it lands mid-audit); --help executes the verb on 14/20
    verbs, so capture --help commits to the memory git while you are screen-sharing; vault --redacted
    still is not sponsor-safe; and the eval gate passes a NO-SHIP verdict with 7 false approves.

Every blocker was independently reproduced on a1d1ec1 before it was written down. The #71 test-plan
verification (4 PASS / 5 PARTIAL / 0 FAIL) is a comment on that PR.

Checks

  • npm run check passes (130 tests; the gate rejected em dashes, fixed in a follow-up commit)
  • No client names, .fde/ exports, credentials, or screenshots with real people - the three
    engagements are fictional clients on real public repos of yours

Link to Devin session: https://app.devin.ai/sessions/04a810382acd4dd794f383beceb4f546
Open in Devin Desktop: https://app.devin.ai/desktop/session/04a810382acd4dd794f383beceb4f546?variant=devin
Requested by: @suboss87

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant