Skip to content

test(desktop): cover the Plan execution lifecycle end to end - #5447

Open
Zestful-ss wants to merge 2 commits into
apache:mainfrom
Zestful-ss:fix/desktop-plan-execution-e2e
Open

Zestful-ss wants to merge 2 commits into
apache:mainfrom
Zestful-ss:fix/desktop-plan-execution-e2e

Conversation

@Zestful-ss

Copy link
Copy Markdown
Contributor

Summary

The Plan execution lifecycle can now be driven end to end in the real Desktop
application, and one spec covers it: Plan mode → proposal → approve → real
progress
→ Stop → full Electron restart → resume → completion.

Two things were missing. The lower tier already tests the Plan panel
(src/main/__tests__/plan-mode-panel-*.test.ts), but it fakes window.maka and
the DOM, so nothing showed that an interrupted execution survives an application
restart or that Stop during a live Turn settles through RootTurnCoordinator
into a durable interruption. And the Desktop E2E harness could not produce a Plan
at all: FakeBackend has no Plan support, and a main-session backend never
receives a tool list — BackendFactoryContext.tools is filled only for subagent
activations, where it is a permission ceiling, not the model's tools.

The fixture invokes the production Plan tool implementations against the
composed Plan store.
The candidate composition reports the Plan authority it
composes through one optional, production-inert observer, and the test-only
backend calls buildSubmitPlanTool / buildUpdatePlanTool through it and runs
their .impl(), so a fixture write takes the production validation and
operation-id path. It does not go through the Runtime's tool discovery or
dispatcher, which is why the fixture reads its Turn kind from the store's active
execution rather than from a tool list. The proposal Turn submits a three-step plan; an execution Turn
records one step of progress and then holds the Turn open until the user stops
it, so the interrupt lands on durable progress; a resumed Turn completes the
remaining steps. update_plan is re-bound from the store's active execution on
every Turn, and each logical action keeps a stable tool-call id so a replayed
Turn takes the idempotent recovery path instead of writing twice.

The spec asserts the cross-process boundary, which is the only reason this
belongs in the Electron tier (the reason is recorded in e2e-budget.json): a
Plan execution's progress is written by a live backend Turn and Stop settles it
through the real Host turn lifecycle; the interrupted execution must then be read
back by a fresh renderer over preload/IPC after a complete Electron restart, with
its per-step statuses intact, before the resumed Turn can finish it.

中文概述

Plan 执行生命周期现在能在真实 Desktop 应用里跑通,并由一条 spec 覆盖:
进 Plan 模式 → 提交提案 → 批准 → 真实进度 → 停止 → 完整退出并重启
Electron 应用
→ 恢复执行 → 完成。

原本缺两样东西。低层已经有 Plan 面板测试,但它们伪造 window.maka 与 DOM,
所以没有任何测试能证明「被中断的执行能在应用重启后存活」,也证明不了「运行中的
Turn 被 Stop 时会经 RootTurnCoordinator 落成持久的中断」。同时 Desktop E2E 的
假模型完全无法产生 PlanFakeBackend 没有 Plan 能力,而主会话后端根本拿不到
工具表(BackendFactoryContext.tools 只对子代理激活填充,那是权限上限,不是模型
工具表)。

fixture 调用生产 Plan 工具实现:候选组合通过一个可选、对生产无影响的 observer
把组合出的 Plan 权威暴露出来,test-only 后端经它调用真实的
buildSubmitPlanTool / buildUpdatePlanTool.impl(),因此 fixture 的写入同样
经过生产校验与 operation-id 路径(它不经过模型的工具发现与 Runtime 派发)。提案 Turn
提交三步计划;执行 Turn 记录一步真实进度后把 Turn
挂住,直到用户停止 —— 这样中断落在已持久化的进度上;恢复后的 Turn 完成剩余步骤。
update_plan 每回合都从 store 的活跃执行现绑定,且每个逻辑动作保持稳定的
tool-call id,重放时走幂等恢复路径而不是重复写入。

spec 断言的是跨进程边界(这也是它属于 Electron 层的唯一理由,已记入
e2e-budget.json):计划进度由运行中的后端 Turn 写入、Stop 经真实 Host 回合
生命周期落定;被中断的执行必须在完整退出并重启 Electron 应用后(新 main
process、新 renderer、重开同一份 SQLite 状态)由全新的渲染进程经 preload/IPC
读回,且逐步状态原样保留,恢复后的 Turn 才能把它做完。

Verification

  • RED, before the fixture capability existed (fixture disabled, spec unchanged):
    the spec fails at its first Plan assertion —
    Locator: locator('.plan-proposal-card[data-status="pending_approval"]') Error: element(s) not found
    which also shows the Plan-mode entry path (composer + menu → mode chip) already worked.
  • RED, targeted mutation on the final tree: with only the sentinel literal in the
    built test-only module renamed, so the fake model no longer submits a proposal, the
    spec fails (1 failed); the module was then restored byte-identically
    (Get-FileHash equal). This is the parent's independent reproduction, not the
    author's.
  • GREEN, repeat runs on this branch's base (origin/main at c980b93a):
    npx playwright test --config e2e/playwright.config.ts e2e/plan-execution-lifecycle.spec.ts --repeat-each=5 --workers=1
    5 passed (1.4m) (18.9s, 16.0s, 16.8s, 15.2s, 15.6s); raw log in
    _local-evidence/pr-desktop-plan-e2e/verify-plan-repeat5.log, preceded by single
    green runs of 17.1s and 14.1s.
  • One environment failure, not in this spec's legs. An earlier --repeat-each=3
    batch was 2 passed / 1 failed, and the failure was
    electronApplication.firstWindow: Timeout 30000ms exceeded while waiting for event "window"
    in withE2eWindow — fixture setup, before any Plan assertion. app.firstWindow()
    uses Playwright's fixed 30 s default, independent of this config's 60 s test timeout,
    so that run's app simply took longer than 30 s to open its first window. In the same
    session the untouched, pre-existing session-local-recovery.spec.ts was also
    2 passed / 1 failed, with the known Windows teardown flake
    (EBUSY: resource busy or locked, unlink …\DIPS), so this is fixture-machine
    flakiness rather than a property of the new spec.
  • On a failed .last-run.json a reviewer may find: it belongs to a run invoked
    without --config (Playwright defaults: 30 s timeout, outputDir
    apps/desktop/test-results), where the restart leg's relaunch hit the dev-profile
    single-instance dialog (Maka Dev 已在运行) because the 30 s test timeout had already
    begun tearing the first instance down. The config-based record — the one npm run e2e
    writes, apps/desktop/e2e/test-results/.last-run.json — is passed. Electron specs
    must be invoked with --config e2e/playwright.config.ts.
  • npm run check:e2e-budgetE2E budget holds: 38 tests in 22 files. (was 37/21)
  • node --test packages/runtime-host/dist/__tests__/desktop-e2e-execution.test.js → 2/2
  • npm run check:staledist is fresh.; npm run lint, npm run format:check → clean;
    npm run typecheck → no error TS.

The full E2E suite was not run (22 files, many minutes). Two neighbouring specs were
re-run as a fixture regression check for the shared fixture code this PR touches:
streaming-remount.spec.ts and expected-failure-feedback.spec.ts → 5 passed, plus
the same teardown-only EBUSY …\DIPS failure documented above.

Review focus

  • The one production change: an optional observePlanStore field on
    ExecutionRuntimeHostCompositionDependencies and one call site immediately after
    the composed Plan authority. Unset means no extra work and no new error path; the
    Desktop E2E composition is its only consumer, in the same spirit as a candidate
    entrypoint that selects a test-only entry module.
  • A deliberate deviation from an obvious assertion: the final leg does not assert a
    rendered 3/3 步. PlanExecutionPanel renders active ?? lastInterrupted, and the
    completion event clears activeExecutionId in the same store transition that
    records the last step, so the panel — its count included — unmounts in that commit.
    The leg asserts panel absence plus the terminal projection
    ({ activeExecutionId: null, status: 'completed', completed: 3, total: 3 }) through
    the same getPlanState preload read the panel uses, rather than inventing a DOM
    state the product never renders.
  • A trade-off to weigh: the spec restates the sentinel literal
    ('__e2e_plan_proposal__') instead of importing it, because that test-only module
    has no package subpath and adding one changes the release-manifest policy test's
    expected omission list. A divergence fails the spec at its first assertion rather
    than passing quietly; happy to add the subpath if you prefer the import.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: ChatGPT planned the work and reviewed it against the code;
DeepSeek Harness (DSH) implemented the fixture and the spec, including a delegated DSH
subagent for the iteration, and ran every check quoted above. The human contributor
directed the work and reviewed the change. Both commits carry a
Generated-by: DeepSeek Harness trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck, the E2E budget gate and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes
  • No production runtime behavior change — the change adds one optional test-only
    composition observer that production callers leave unset.

…ifecycle

The Desktop E2E fixture could not produce a Plan at all: `FakeBackend` has no
Plan support, and a main-session backend never receives a tool list —
`BackendFactoryContext.tools` is filled only for subagent activations, where it
is a permission ceiling, not the model's tools. A spec could therefore never see
a proposal, let alone an approved execution.

`DesktopE2eBackend` now drives the real Plan tools through the very store the
Host builds them from. The candidate composition reports its composed Plan
authority through one optional, production-inert observer, and the test-only
module calls the real `buildSubmitPlanTool` / `buildUpdatePlanTool` builders, so
the writes take the production validation and operation-id path. The proposal
Turn submits a three-step plan; an execution Turn records one step of progress
and then holds the Turn open until the user stops it, so an interrupt lands on
durable progress; a resumed Turn completes the remaining steps. `update_plan` is
re-bound from the store's active execution on every Turn, and each logical action
keeps a stable tool-call id so a replayed Turn takes the idempotent recovery path
instead of writing twice.

Generated-by: DeepSeek Harness
The Plan execution lifecycle had no Electron coverage, and the panels' own tests
live in a lower tier: `src/main/__tests__/plan-mode-panel-*.test.ts` fake
`window.maka` and the DOM, so none of them can show that an interrupted execution
survives an application restart, or that Stop during a live Turn settles through
`RootTurnCoordinator` into a durable interruption.

The new spec enters Plan mode through the composer's own menu, approves the
fixture proposal, asserts acquired progress (`1/3 步`, `正在执行计划`), stops the
live Turn and asserts the interruption preserves that progress, restarts the whole
Electron application and asserts a fresh renderer reads the same per-step
statuses back over preload/IPC, then resumes and asserts the terminal state.

`PlanExecutionPanel` renders `active ?? lastInterrupted`, and the completion event
clears `activeExecutionId` in the same store transition that records the last
step, so the panel unmounts with it: the final leg asserts panel absence plus the
terminal projection through the same preload read the panel uses, rather than a
`3/3 步` label the product never renders.

Electron-owned mechanism, as recorded in `e2e-budget.json`: a Plan execution's
progress is written by a live backend Turn and Stop settles it through the real
Host turn lifecycle; the interrupted execution must then be read back by a fresh
renderer over preload/IPC after a complete Electron restart, with its per-step
statuses intact, before the resumed Turn can finish it.

Generated-by: DeepSeek Harness
@github-actions github-actions Bot added the effort/M Under 500 readable lines label Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/M Under 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant