Skip to content

th-e77603: declarative scrape rules + aider/goose/crush/cline harnesses - #599

Merged
brentrager merged 7 commits into
mainfrom
th-e77603-harness-scraped
Sep 14, 2026
Merged

brentrager merged 7 commits into
mainfrom
th-e77603-harness-scraped

Conversation

@brentrager

@brentrager brentrager commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Pearl th-e77603 (epic th-faa590) · fixes th-d2a1e4

Problem

SmoothFlow could only learn state for harnesses with hooks. Hookless coding CLIs (aider, goose, crush, cline…) show idle / working / needs-you only on their terminal, and the flat [state.scrape] lists can't describe real TUIs: crush draws its permission modal over a footer that still says esc cancel, cline's composer looks idle for the whole turn, aider streams answers with no marker at all. And prompt_as = "paste" pasted on a 4 s timer, straight into aider's first-run question.

Solution

  • [[state.scrape.rules]] (crates/smooth-flow/src/scrape.rs): ordered rules, first match wins, evaluated before the flat lists (which keep their exact precedence). A rule is match regexes over a scope (tail / pane / last_line / cursor_line / title) with all, unless, unless_below, quiet_ms, changed_within_ms, alternate_screen, spinner. Pure over a PaneObservation; ANSI is stripped first.
  • Engine: one display-message per tick for title / alt-screen / cursor row, a per-session "pane last changed" clock, the deciding rule's name in the needs-you detail, a scraped needs-you clears on the next idle, and paste waits for a scraped idle (never into a needs-you; 90 s fallback; manifests with no idle signal keep the 4 s timer).
  • resume.mode = "continue_latest" for CLIs that only continue "the latest conversation here".
  • Built-ins: aider, goose, crush, cline.
  • Docs: Harness-Manifests.md — rule semantics + load/timing guidance, the shipped table (live vs fixture-only), how cmux and orca detect hookless state (file:line), and why Claude's detect.rs is expressible as rules.

Verification

  • Live, real CLIs through the engine (tests/scrape_live.rs, #[ignore]), fresh scratch $HOME, isolated installs, local mock OpenAI server (no accounts, no keys): aider 0.86.2, goose 1.50.0, crush 0.94.2, cline 3.0.61 each went launch → first-run needs-you → working → idle, steer → working → idle, tool/edit request → needs-you → deny → idle, kill + resume → back.
  • Fixture-only: auggie 0.36.0 and kiro-cli 2.21.4 stop at sign-in walls (not pressed); wall captures are committed and tested never to read idle/working under any built-in. No built-in manifest for either.
  • 44 real pane fixtures + tests/scrape_fixtures.rs (verdict + deciding rule per fixture, resize/padding/SGR invariants, time monotonicity, detect.rs-as-rules equivalence on 5 000 generated panes). Unit/property tests in scrape.rs, harness.rs, engine.rs, plus a live fake-CLI test for the paste gate and needs-you clearing.
  • cargo test -p smooai-smooth-flow green (168 + 6), cargo clippy -p smooai-smooth-flow --all-targets has no warnings in the files this PR touches. Not run locally: smooth-daemon / smooth-cli test builds (disk under 6 GiB on the build machine). Their only changes are list assertions that now count BUILTIN rather than assuming four, so CI is the check for those.

Harness conformance (#598) — expected failure

tests/conformance/{aider,goose,crush,cline}.toml enrol the four in the conformance suite, with screens cut from the live captures. Once this PR is rebased onto #598, screen_problems gets patched here to count [[state.scrape.rules]] and to feed painted screens a quiet time. Until follow-up th-5a2314 (harness-conformance: manifest-driven [steer] approve_keys / deny_keys) lands, the needs_you → approve step fails for all four. The engine approves with keystroke 1; aider wants y/Enter, goose Enter, crush Enter, cline y. This is expected, not a regression.

🤖 Generated with Claude Code

brentrager and others added 5 commits September 14, 2026 17:29
…oose, crush, cline) with live pane fixtures

WIP checkpoint: [[state.scrape.rules]] engine (crate::scrape), engine
observation (title/alt-screen/cursor/quiet clock), paste waits for a scraped
idle (th-d2a1e4), resume.mode = continue_latest, 44 real pane fixtures.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e-CLI tests, clippy

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…_latest keeps pasted-launch flags, docs

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ing four

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Sep 14, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 6200ada

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@smooai/smooth Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@brentrager
brentrager enabled auto-merge (squash) September 14, 2026 21:32
brentrager and others added 2 commits September 14, 2026 17:33
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…the live captures

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@brentrager
brentrager merged commit 82930b5 into main Sep 14, 2026
4 checks passed
brentrager added a commit that referenced this pull request Sep 17, 2026
…en (#602)

Version-only bump. Brent is running 0.2.2, and the two features he is
waiting on have been sitting on main unreleased: the New Session screen
that no longer demands a pearl id (#594) and the declarative scrape rules
that add aider/goose/crush/cline plus the aider paste-timing fix (#599).
Cutting 0.2.3 is what actually puts them in front of a user.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant