docs: wave 2 field test on 3.16.1 - developer report + three run reports - #72
Open
devin-ai-integration[bot] wants to merge 2 commits into
Open
docs: wave 2 field test on 3.16.1 - developer report + three run reports#72devin-ai-integration[bot] wants to merge 2 commits into
devin-ai-integration[bot] wants to merge 2 commits into
Conversation
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Wave 1 (#70) covered about half the method and found three surfaces that volunteered false statements to
a sponsor. You patched those in #71. Wave 2 was built to close the coverage gap and to attack the fixes
rather than confirm them: three FDEs, three engagements on three real client repos, entered cold from the
README, assigned so that all 37 references, every CLI verb and every adapter had a named owner - including
the 15 references nobody had touched (the whole commercial/exec path) and the never-driven LOCAL-LLM
adapter.
Based on
field-gates(3.16.1 @a1d1ec1), notMain, because that is the revision under test and theonly one where
npm run checkis green.What changed
Docs only - four field reports, no source touched:
docs/field-reports/2026-08-28-wave2-developer-report.md- the consolidated report: verdict, thefixed-the-instance-not-the-class analysis of 3.16.1, defects ranked blocker → papercut with exact
command output, 37-reference coverage with per-artifact send/edit/never verdicts, and eight ordered
changes.
exec-commercial-health,deep-engineering-localllm,lifecycle-redteam).Headline, so you can judge whether the rest is worth reading:
@fderouted to the right reference unprompted in 48/51 exercised cases, and the untestedbusiness-case → board-memo → red-team chain works end to end - it found two real gaps in an FDE's own
board memo.
mounts, missing git identity), and the
<private>sentinel appeared in 0 of 12 output surfaces.still mints a stakeholder (
batch,still,Friday,Dr,Zo); two people sharing a first namecollapse so one Will's green clears the other's amber;
.last-writekeeps the secret afterredact --apply; an emptyreality.mdwarns nothing.receiptsasserts "nothing was ever logged" over an entry sealed by an unclosed<private>(same class as the wave-1 bugs, and it lands mid-audit);--helpexecutes the verb on 14/20verbs, so
capture --helpcommits to the memory git while you are screen-sharing;vault --redactedstill is not sponsor-safe; and the eval gate passes a
NO-SHIPverdict with 7 false approves.Every blocker was independently reproduced on
a1d1ec1before it was written down. The #71 test-planverification (4 PASS / 5 PARTIAL / 0 FAIL) is a comment on that PR.
Checks
npm run checkpasses (130 tests; the gate rejected em dashes, fixed in a follow-up commit).fde/exports, credentials, or screenshots with real people - the threeengagements are fictional clients on real public repos of yours
Link to Devin session: https://app.devin.ai/sessions/04a810382acd4dd794f383beceb4f546
Open in Devin Desktop: https://app.devin.ai/desktop/session/04a810382acd4dd794f383beceb4f546?variant=devin
Requested by: @suboss87