th-3cabf6: harness conformance contract + th harness doctor - #598
Open
brentrager wants to merge 3 commits into
Open
brentrager wants to merge 3 commits into
brentrager wants to merge 3 commits into
Conversation
…ent speaking its mechanism A generic fake agent (smooth-flow-fake-agent) derives argv parsing and hook events from the manifest itself, so a new harness enrolls by being listed in BUILTIN (scrape ones add captured screens). The rig runs resolve · launch · working · idle · steer · permission · resume · kill on a private engine with an in-process hook listener and a cmux-shim decoy first on PATH. It caught a real engine bug: a permission ask under a harness's own event name (event_map → needs_you) carried no request_id, so flow.approve / th flow approve could not address it. Mapped permission asks now get a hook-<uuid> id that falls through to the approval keystroke. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s machine Read-only per-harness verdict (works / degraded / not installed) with the one command that fixes each degraded row: the binary SmoothFlow runs and whether `which` returns a cmux shim, --version, resolution under the SmoothFlow app's launchd PATH (including #!/usr/bin/env scripts whose interpreter the app cannot see), hooks installed AND trusted (Codex's Hooks-need-review state, stale smooth-agent plugins without the flow hook, OpenCode plugins without the generic event hook), daemon reachability and sign-in. --json for the app. Plus the Harness conformance CI job and the 'Supporting a harness' contract docs. Picker surfacing is filed as th-51bf88 (needs the doctor core in the daemon and all four pickers — not small). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: cc9337e The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
brentrager
enabled auto-merge (squash)
September 14, 2026 21:02
brentrager
added a commit
that referenced
this pull request
Sep 14, 2026
Both lanes fixed un-approvable hook asks. Adopt #598's rule (id assigned in the pure mapper, permission asks only; a question gets none) so the two merge cleanly. The delta kept here: the permission label is derived from the event name / notification_type, and only Claude-protocol harnesses hold PermissionRequest open. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nks; oxfmt the contract doc Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
"Broader harness support" (sibling lanes are adding ~13 harnesses) is only worth something if a user can tell which harnesses actually work on their machine, and a regression is caught. Today neither is possible short of launching one and watching.
Solution
1. Conformance contract (
crates/smooth-flow): every manifest inharness::BUILTIN(+ reference manifests intests/conformance/manifests/) runs resolve · launch · working · idle · steer · permission · resume · kill againstsmooth-flow-fake-agent, a stand-in CLI that derives its argv parsing (match_argv, the inverse ofrender_argv) and its hook events (Claude table / invertedevent_map/ scraped screens) from the manifest itself. Private engine per manifest: scratch HOME/flow.db/worktree, own tmux socket, in-process/api/flow/hookslistener with the permission long-poll, binary resolution pinned to the scratch HOME with a cmux shim decoy first on PATH. No real CLIs, network or credentials. A new harness enrolls by being listed inBUILTIN; scraped ones add screens captured from the real CLI, checked against their own regexes.2.
th harness doctor [name] [--json] [-v](crates/smooth-cli/src/harness_doctor.rs): read-only verdict works / degraded / not installed + the one fix per degraded row — binary actually launched vs cmux shim,--version, resolution under the SmoothFlow app's launchd PATH (incl.#!/usr/bin/env nodescripts), hooks installed and trusted (Codex[hooks.state]), stale smooth-agent plugin without the flow hook, OpenCode plugin without the genericeventhook, daemon reachability, sign-in (env / files / keychain item presence).3. CI: new
Harness conformancejob (Linux, tmux,SMOOTH_E2E_STRICT=1).4. Engine bug the suite found on its first run: a permission ask under a harness's own
event_mapname carried norequest_id, soth flow approve/ the apps could not approve it. Now getshook-<uuid>→ falls through to the approval keystroke. PlusEngine::set_resolve_envtest seam.Picker surfacing is not small (needs doctor core in the daemon + 4 pickers per the mobile parity rule) → filed th-51bf88.
Verification
cargo test -p smooai-smooth-flow: 159 lib tests + conformance suite green (~23 s): claude, opencode, codex, th-code, conformance-mapped, conformance-scrape all 8/8 (th-code permission n/a).cargo test -p smooai-smooth-cli: 846 + 3 green (incl. 11 doctor tests + thedoctorclap parse test);cargo clippy -p smooai-smooth-cliadds no warnings (one pre-existing in harness.rsadd). Realth harness doctorrun from the built binary matches below.th harness enable claude-code;which claudeis a cmux shim, correctly skipped), opencode degraded (linked plugin posts no flow events →th harness enable opencode), codex works (7 flow hooks, all trusted), th-code works.Pearl th-3cabf6 (epic th-faa590).
🤖 Generated with Claude Code