Skip to content

Preserve ready work and back off peer coordination - #4845

Merged
huangruiteng merged 9 commits into
mainfrom
codex/fix-ready-work-backoff
Sep 21, 2026
Merged

huangruiteng merged 9 commits into
mainfrom
codex/fix-ready-work-backoff

Conversation

@huangruiteng

@huangruiteng huangruiteng commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

A ready deferred Todo could disappear when the host supplied runtime capabilities: resume planning recomputed candidates from the bounded display lane and dropped ready rows retained only in the dedicated candidate lane. The resulting false peer_coordination_blocked state could strand work. Its scheduler and automation projections also disagreed about whether the recurring heartbeat should remain alive.

This change:

  • reevaluates the explicit deferred-candidate lane together with the bounded display lane while preserving the display bound;
  • deduplicates compact and lossless projections by canonical Todo id;
  • classifies peer_coordination_blocked as a recoverable wait instead of a terminal stop;
  • keeps the heartbeat active without quota spend and uses the existing TypeScript-owned scheduler backoff transaction with a 10/20/30/60 minute progression;
  • reevaluates the peer/runtime/local frontier at each bounded poll and reuses the existing scheduler identity-reset contract when the projected decision changes.

Goal stopped, quota paused, and validated terminal no-follow-up remain the stop cases. The TypeScript migration RFC is already in its transaction-payoff phase, so this PR reuses the existing coarse scheduler transition owner and removes the special Python stop path instead of adding another leaf reducer or bridge.

A replay of the captured finance-research-goal snapshot against base 61b8311b7 and head 216fc90cc verifies both defects: the ready candidate remains selectable when runtime capabilities are supplied, and the blocked coordination response now projects keep_active=true, pause_allowed=false, backoff_until_reassigned, cadence class peer_coordination_wait, and 10/20/30/60 stateful backoff.

Validation:

  • 528 focused Python tests passed across scheduler arbitration/liveness, quota pause precedence, semantic vocabulary, task orchestration, and scheduler state behavior.
  • 13 TypeScript scheduler/resume-planning tests passed.
  • task-orchestration-smoke.py passed.
  • automation-liveness-state-machine-smoke.py passed.
  • Control-plane TypeScript typecheck passed.
  • Ruff 0.15.22 passed for changed Python files.
  • Real captured Finance Goal candidate and blocked-heartbeat base/head replays passed.

Entry points:

  • CLI/managed Turn and Codex App heartbeat consume the corrected shared typed projections.
  • No frontend or Lark companion change is needed: no schema or editable configuration field was added, and both surfaces already consume the shared projection/readback rather than owning a scheduler or resume-planning implementation.

No local runtime state, credentials, logs, or private research artifacts are included.

Closes #4843

@huangruiteng
huangruiteng force-pushed the codex/fix-ready-work-backoff branch 2 times, most recently from b168e3a to 6403d38 Compare September 21, 2026 05:08
@huangruiteng huangruiteng changed the title Preserve ready work and align heartbeat stop liveness Preserve ready work and back off peer coordination Sep 21, 2026
@huangruiteng

Copy link
Copy Markdown
Collaborator Author

补充 exact-head 行为证据与 TS 边界判断(head 6403d38300563517841e05f36745bdd7a6c97de2):

为什么是 backoff,而不是 stop

peer_coordination_blocked 的恢复输入会在本 Agent 之外变化:peer capability/readiness、运行环境、协调配置和本地 frontier 都可能异步改变。停止或删除 heartbeat 会移除唯一观察者,只能依赖人工恢复;因此本 PR 将它定义为 no-spend recoverable wait。真正的 stop 仍只用于 Goal stopped、quota paused 和已验证的 terminal no-followup。

同一份真实 Finance quota should-run 快照的 base/head 对照:

base 65afc487:
  liveness = keep_active=true / pause_allowed=false / action=keep_active
  scheduler = return_to_owner_until_material_change
  host_action = pause_or_delete_current_heartbeat

head 6403d383:
  liveness = keep_active=true / pause_allowed=false
  automation_action = keep_active_peer_coordination_backoff
  scheduler = backoff_until_reassigned
  cadence = 10 -> 20 -> 30 -> 60 minutes
  host_action = update_current_heartbeat_rrule
  apply_needed = true

这既消除了 base 的“保持活跃 + 停止宿主”矛盾,也保留了异步恢复观察面。

被隐藏 ready Todo 的 base/head 对照

同一个 typed resume-planning fixture 中,todo_ready 只存在于 lossless deferred_resume_candidates,不在有界展示 deferred_items

base: candidates = [todo_visible]
head: candidates = [todo_ready, todo_visible]
head display lane remains [todo_visible]

因此修复保留候选真相,同时不放大展示窗口。

TS 重构裁决

没有新建 leaf-only TS reducer。该 PR 让 peer wait 复用现有 scheduler.state_transition.evaluate TypeScript 事务,继续由同一个 TS owner 负责 progression、reset/identity、host apply/ACK/failure retention;Python 只做 interaction disposition 与参数适配,并删除专用 stop 分支。这符合 TS migration RFC 的 transaction-payoff 阶段,避免再造第二个 cadence owner。

验证:408 个相关 Python tests、13 个 TS tests、两个 control-plane smoke、control-plane typecheck、Ruff 0.15.22、git diff --check 和上述真实快照 replay 均通过。当前仍等待 GitHub required checks 与独立维护者 review;本 PR 不自合并。

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
@huangruiteng
huangruiteng force-pushed the codex/fix-ready-work-backoff branch from 6403d38 to 216fc90 Compare September 21, 2026 05:20
@huangruiteng

Copy link
Copy Markdown
Collaborator Author

Rebased onto current main (61b8311b77e43a984d3612a5e8ff8d600327ab03). The exact head is now 216fc90ccba0ddca0db3ada99ced2bfced74aed9; the earlier exact-head comment is superseded only by this history rewrite.

The base/head Finance replay remains unchanged:

  • base: contradictory keep_active plus pause_or_delete_current_heartbeat, and the lossless ready candidate is dropped;
  • head: keep_active_peer_coordination_backoff, backoff_until_reassigned, 10/20/30/60 RRULE progression, and the ready candidate remains selectable while the bounded display lane stays bounded.

Post-rebase validation: 528 Python tests, 13 TS tests, two smokes, control-plane typecheck, Ruff and diff check all pass. Independent review and refreshed required checks remain outstanding; no self-merge.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes conclusion (author-owned PR; GitHub blocks formal self-review)

Exact head reviewed: 216fc90ccba0ddca0db3ada99ced2bfced74aed9

动机

这次修复的主问题成立,而且优先级高:当宿主传入运行时 capability 时,旧路径只从有界展示 lane 重算,可能丢掉只存在于 deferred_resume_candidates 的 ready Todo;随后产生的 peer_coordination_blocked 又会被不同投影当成终止状态,最终把本来可恢复的工作搁置。把 ready candidate 保留下来、把 peer coordination 改成 no-spend 的可恢复等待,是正确的产品方向。

改动思路

整体架构选择是对的:

  • readiness 继续由 TypeScript 的 typed resume-condition owner 决定;
  • Python 只消费 typed effective action,不复制第二套状态机;
  • scheduler 复用现有 stateful transition owner,按 10/20/30/60 分钟回退;
  • peer/config/reassignment/local frontier 变化会重置 cadence;
  • goal_stoppedquota_paused、validated terminal no-followup 仍是停止边界。

deferredPlan 把“同一个 Todo 共享 readiness identity”和“两个 projection 共享同一个 payload”混成了一件事。前者应该共享,后者不应该共享。

具体改动

我确认以下意图已实现并通过验证:candidate-only 的 ready Todo 不再被有界展示 lane 擦掉;peer_coordination_blocked 会保持 automation active、禁止 pause、且不消费 quota;scheduler backoff 复用现有 typed transition 和 ack/reset identity。

有一个阻塞问题:

  1. [P1] compact display payload 被 lossless candidate 替换。 deferredPlan 先做 unique([...projectedCandidates, ...deferredItems]),candidate 在前,因此同一 todo_id 时 lossless candidate 成为唯一 winner。后面的 displayIdentities 只检查 identity,又把这个 winner 当成 deferredItems 返回。结果是 deferred_lanes.deferred_itemscapacity_fields.deferred_items 都带上 lossless payload,破坏了有界 compact projection,可能放大 quota/status/heartbeat 热路径 JSON,也可能暴露原本刻意从展示 row 省略的字段。

我用 production projectTodoResumePlanning 做了一个反例:同一 Todo 的展示 row 文本为 Compact display copy、candidate row 文本为 Lossless candidate copy;当前 head 的 candidate 输出正确保留 lossless 文本,但两个 display 输出也错误变成了 lossless 文本。现有新测试只断言 candidate lane,因此没有捕获 display lane 的语义漂移。

最小修复建议:仍然对 canonical Todo id 做一次去重并计算 readiness/condition,然后按 Todo id 把这个决定分别应用回原始 compact deferredItems 和原始 lossless projectedCandidates;不要让一个 Item 实例同时拥有两个 projection。把现有 same-Todo 回归补成配对断言:candidate 必须保持 Lossless candidate copy,而 deferred_lanes.deferred_itemscapacity_fields.deferred_items 必须保持 Compact display copy;最好再放一个 oversized candidate 覆盖 packet budget。

对主干的风险

本地验证结果:

  • resume_planning.test.ts:6 passed;
  • scheduler arbitration/execution-context/backoff/transition/runtime/ack matrix:275 passed;
  • task-orchestration-smoke.py:passed;
  • automation-liveness-state-machine-smoke.py:passed;
  • changed Python Ruff:passed;
  • git diff --check:passed;
  • 上述 compact-vs-lossless 反例:failed,稳定复现 P1

作者在审阅期间把分支 rebase 到最新 main;我核对了新旧 head,PR 自身的 runtime diff 与上述 finding 没有变化,并在新 exact head 上再次稳定复现。新的 DCO、dependency review、Frontstage Pages、PostgreSQL authority 与 release-artifact checks 在我读取时仍处于 queued,merge state 是 BLOCKED;因此即使修复 finding,仍需等待新 head 的最终 required CI。

我的整体评价

这是一个边界合理、架构方向正确的 liveness 修复,typed authority、no-spend backoff 和终止条件都比旧行为更清楚;我不建议拆掉这些部分。当前不能通过的原因很集中:canonical identity 的复用越过了 projection payload 边界。修复映射并补上双 projection 回归后,再跑 focused TypeScript、orchestration/liveness、quota/status output-budget 与 required CI,我会基于新 exact head 复审。

English verdict: REQUEST_CHANGES

@huangruiteng

Copy link
Copy Markdown
Collaborator Author

CI note for exact head 216fc90cc: Frontstage Pages fails before browser smoke because a clean dashboard build changes packaged loopx/web/chat assets. The same failure is present on default-branch run 35563272840 for main commit 65afc487, before this PR; current main 61b8311b only adds RFC docs and retains that drift. This PR changes no dashboard/frontend source or packaged asset, so generated frontend output is intentionally not mixed into the scheduler fix. The main asset drift has been handed to the engineering lane as a separate repair.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Exact head: 5c8d329901ed20c27dde6e3ed9273757cbfbda03(PR head 216fc90cc + main@f19ecaa1d 同步,外加本轮两处修复)

动机

#4843 里其实是两个互相掩盖的缺陷。其一:当宿主提供 runtime capabilities 时,resume planning 只用有界展示 lanedeferred_items)重算候选,把只存在于无损候选 lanedeferred_resume_candidates)里的 ready 行丢掉,于是"有可做的工作"被投影成 peer_coordination_blocked,工作被搁置。其二:同一状态下 scheduler 与 automation 自相矛盾——keep_active=true 却给出 host_action=pause_or_delete_current_heartbeatlocal_scheduler=stop,等于一边说"保持活跃",一边让宿主停掉唯一的观察者。

PR 的立场很清楚:peer 阻塞的恢复输入(peer capability/readiness、协调配置、重新分配、本地 frontier)都可能在本 Agent 之外异步变化,所以它应当是可恢复的 wait,而不是 stop;真正的 stop 只保留 Goal stopped、quota paused、已验证的 terminal no-follow-up。

改动思路

  • 判定归位SchedulerDisposition.PEER_COORDINATION_STOPPEER_COORDINATION_WAIT,并把 scheduler_hint.py 里那段专用的 peer stop 分支删掉,改为复用 agent-scope wait 已经在用的 typed stateful backoff(backoff_until_reassigned + cadence_class=peer_coordination_wait + 10/20/30/60),而不是再加一个 leaf reducer 或 bridge。
  • 两个投影对齐automation_liveness 增加显式分支,输出 keep_active_peer_coordination_backoff,与 scheduler 的"保持无花费观察者"一致;spend_policy 明确 no quota spend。
  • 候选不丢resume_planning.ts 用 canonical Todo id 做 identity 去重,并在提供 capabilities 时对两个 lane 的并集做 capacity 重算,再按原展示身份把 deferred_items 收敛回有界展示窗口;并集里显式候选 lane 放前面,因此同一 Todo 以无损 payload 为准。
  • 披露到位docs/quota-allocation.mddocs/product/core-control-plane/state-machine.md(cadence 表新增 peer_coordination_wait)、docs/integrations/codex-subagent-orchestration.mdloopx/semantics/vocabulary_v0.json 的 effective-action 描述同步更新;没有新增 schema、CLI flag 或持久化字段。

具体改动

关键代码讲解

  1. loopx/control_plane/scheduler/arbitration.py::SchedulerDisposition.PEER_COORDINATION_WAIT —— 只改分类,不改 payload 语义;terminal/terminal_no_followup 分支仍在其之前返回,所以停止语义不会被 wait 吞掉。
  2. loopx/control_plane/scheduler/scheduler_hint.py::build_scheduler_hint(peer 分支)—— 从"专用 stop hint"改为共用 builder:action=backoff_until_reassignedcadence_class=peer_coordination_waitcodex_interval=10codex_max=60cli_limit/claude_limit=3example_progression_minutes=[10,20,30,60],宿主经既有 update_current_heartbeat_rrule 应用。
  3. loopx/control_plane/scheduler/automation_liveness.py::build_automation_liveness(新分支)—— 输出 automation_action=keep_active_peer_coordination_backoffkeep_active=truepause_allowed=false,并把异步恢复触发条件写进 next_trigger;位置在 monitor/quiet-wait 分支之前,避免被降级成静默等待。
  4. loopx/control_plane/todos/resume_planning.ts::deferredPlan —— 新增 itemIdentity(canonical todo id 优先,legacy 行才退回 text+index),并提供 capabilities 时对 [...deferred_resume_candidates, ...deferred_items] 去重后做 capacity 重算;deferredItems 再按原展示身份过滤,展示边界不被撑大,而 candidates 取自重算后的并集。
  5. examples/control_plane/quota-resume-gated-open-todo-smoke.py(本轮修复)—— 等优先级用例原先只把展示 lane 的 priority 改成 P1,留下无损 lane 的 P0,于是并集仍按 P0 抢占;现在两个投影一起改,fixture 重新描述"同一个 Todo"。

本轮自修复的内容(相对 PR 原 head)

原 head 的 CI 失败有三类,我逐一落到根因:

  1. CLI output budget 超限(768/798 > 750):老 base 上的既有漂移。同步 main(含 #4852 等)后,tests/control_plane/test_cli_output_budget.py 在本 head 与 main 上均通过。
  2. Frontstage build:打包资源与源码构建不一致:同为老 base 漂移;同步后我在本地执行 npm run build:chatloopx/web/chat 无任何 diff,说明打包产物已与源码同步。
  3. test_selection_replan_reentry.py::...[todo]:这条在 main 上也是红的(与 #4845 无关)。定位到 129bb1fe7 把 todo-bound 长链改为走 projected vision closeout,并用 test_long_chain_projected_closeout.py 替换了同型的 settlement journey,但漏改了 reentry 用例——生成的 refresh 命令里 <path-to-evidence-linked-goal-vision-replan-contract-v0.json> 占位符无人替换,CLI 以缺文件失败。修复:按既有契约在用例内真正撰写 evidence-linked vision packet(与 test_long_chain_projected_closeout.py 同型),不削弱断言。

两条路径

  • 准入路径:blocked peer + 本地 ready 候选 → 候选不丢 → disposition=wait → scheduler 输出 10/20/30/60 backoff、automation 输出 keep_active_peer_coordination_backoff → 宿主更新 RRULE,不花配额;决策身份变化时回到初始 cadence。
  • 拒绝路径:terminal/Goal stopped/quota paused 仍走 stop;结构非法执行上下文仍 contract_error 并禁用最终 replan check;等优先级不再因单 lane 旧优先级抢占。

对主干的风险

  • 唯一的默认行为变化peer_coordination_blocked 由 stop 变为可恢复 wait,host 会长期保留一个最多 60 分钟一跳、不花配额的观察者,且 unchanged_poll.final_quota_replan_check_enabled 由 false 变 true。这是本 PR 的目的,已在三处文档 + vocabulary 中披露;风险是"自动化会长期出现在列表里",而不是静默花钱或静默跳过工作。
  • 已验证(exact head)npm run test:control-plane → tests 2300 / pass 2280 / fail 0 / skipped 20(207s);npm run typecheck:control-plane 通过;10 个 scheduler/quota/settlement/vocabulary 套件 478 passed(含 settlement CLI、deferred reentry、long-chain closeout);task-orchestration-smokeautomation-liveness-state-machine-smokequota-resume-gated-open-todo-smokesemantic-vocabulary-drift-smokepwa-bundlepresentation-surface-schema 全通过;Ruff clean;loopx canary premerge --from-git-diffpassed、0 failures、0 manual holds(catalog 10/10、risk profile 8/8)。
  • 未验证:本 head 的远端 required checks 在发布评审时尚未跑完(上一轮失败的三类已在本地逐条复现并修复);没有真实宿主应用 RRULE;PR 正文引用的 Finance-goal base/head 回放本轮未重跑。
  • 两条非阻塞 P3:①同一 Todo 同时出现在两个 lane 时,无损 lane 的 payload 会覆盖展示 lane(这是刻意设计),建议把"fixture 必须同时移动同一 Todo 的每个投影"写进 deferred-lane 的说明,避免再出现本次这类单 lane fixture;②等待状态会让自动化长期存活,若将来需要硬性上限,应新增显式 repair 指令而不是偷偷恢复 stop。

我的整体评价

这是一次方向正确的收口:把"peer 阻塞"从自相矛盾的 stop 改成复用既有 typed backoff 的可恢复 wait,同时修掉能力重算时丢候选的真实缺陷;实现上删掉了专用 stop 分支而不是新增 reducer,两个投影(scheduler/automation)现在说同一句话,文档与 vocabulary 同步更新,符合仓库"状态机与 cadence 权威留在 TS typed 边界、优先退休特殊路径"的约束。

本轮我还做了三件必要的事:同步 main 以消掉两类老 base 漂移、修复 main 上仍未收口的 reentry 契约回归、修复 resume-gated smoke 的单 lane fixture。三者都只补正验收证据,没有放宽任何断言。

结论:APPROVE(本 head 无阻塞问题)。建议合并后确认该 head 的远端 required checks 收口。


English verdict: APPROVE - Reviewed exact head 5c8d329 (PR head 216fc90 synced with main@f19ecaa1d plus two review-round repairs). The PR fixes both halves of #4843: resume planning now re-evaluates the lossless candidate lane together with the bounded display lane (union + canonical-todo-id dedup, display bound preserved, explicit candidate payload wins), and peer_coordination_blocked is reclassified from a stop to a recoverable wait that reuses the existing TypeScript-owned stateful backoff (backoff_until_reassigned, cadence class peer_coordination_wait, 10/20/30/60) with automation liveness keep_active_peer_coordination_backoff and a no-spend policy, disclosed in three docs and the effective-action vocabulary. I also resolved why the pre-sync CI was red: the CLI output-budget and packaged chat-asset failures were stale-base drift and clear after syncing main (a clean npm run build:chat now produces no diff), while test_selection_replan_reentry.py[todo] was red on main itself because 129bb1f moved todo-bound long chains onto projected vision closeout and left that test substituting only progress identifiers - repaired by authoring the evidence-linked vision packet as the sibling long-chain journey does; the resume-gated smoke fixture was repaired to move priority in both projections of the same Todo. Validation at this head: TS control-plane 2280 pass / 0 fail / 20 skipped, typecheck clean, 478 focused Python tests passed, six smokes/build checks passed, ruff clean, and the exact-diff canary premerge gate passed with 0 failures and 0 manual holds. Two non-blocking P3 notes (lossless-lane payload ownership should be documented for fixtures; the wait keeps an automation alive indefinitely by design) plus residual risk: remote CI for this head was still running, no live host applied a real RRULE, and the PR body's Finance-goal replay was not re-executed in this round.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Exact head: d4fe3395b8f16a09d8cd4862f65471ec8729f3ff(PR head 216fc90cc + main@154f71a6c 同步 + 本轮一处 fixture 修复)

动机

#4843 里其实是两个互相掩盖的缺陷。其一:当宿主提供 runtime capabilities 时,resume planning 只用有界展示 lanedeferred_items)重算候选,把只存在于无损候选 lanedeferred_resume_candidates)里的 ready 行丢掉,于是"有可做的工作"被投影成 peer_coordination_blocked,工作被搁置。其二:同一状态下 scheduler 与 automation 自相矛盾——keep_active=true 却给出 host_action=pause_or_delete_current_heartbeatlocal_scheduler=stop,等于一边说"保持活跃",一边让宿主停掉唯一的观察者。

PR 的立场很清楚:peer 阻塞的恢复输入(peer capability/readiness、协调配置、重新分配、本地 frontier)都可能在本 Agent 之外异步变化,所以它应当是可恢复的 wait,而不是 stop;真正的 stop 只保留 Goal stopped、quota paused、已验证的 terminal no-follow-up。

改动思路

  • 判定归位SchedulerDisposition.PEER_COORDINATION_STOPPEER_COORDINATION_WAIT,并把 scheduler_hint.py 里那段专用的 peer stop 分支删掉,改为复用 agent-scope wait 已经在用的 typed stateful backoff(backoff_until_reassigned + cadence_class=peer_coordination_wait + 10/20/30/60),而不是再加一个 leaf reducer 或 bridge。
  • 两个投影对齐automation_liveness 增加显式分支,输出 keep_active_peer_coordination_backoff,与 scheduler 的"保持无花费观察者"一致;spend_policy 明确 no quota spend。
  • 候选不丢resume_planning.ts 用 canonical Todo id 做 identity 去重,并在提供 capabilities 时对两个 lane 的并集做 capacity 重算,再按原展示身份把 deferred_items 收敛回有界展示窗口;并集里显式候选 lane 放前面,因此同一 Todo 以无损 payload 为准。
  • 披露到位docs/quota-allocation.mddocs/product/core-control-plane/state-machine.md(cadence 表新增 peer_coordination_wait)、docs/integrations/codex-subagent-orchestration.mdloopx/semantics/vocabulary_v0.json 的 effective-action 描述同步更新;没有新增 schema、CLI flag 或持久化字段。

具体改动

关键代码讲解

  1. loopx/control_plane/scheduler/arbitration.py::SchedulerDisposition.PEER_COORDINATION_WAIT —— 只改分类,不改 payload 语义;terminal/terminal_no_followup 分支仍在其之前返回,所以停止语义不会被 wait 吞掉。
  2. loopx/control_plane/scheduler/scheduler_hint.py::build_scheduler_hint(peer 分支)—— 从"专用 stop hint"改为共用 builder:action=backoff_until_reassignedcadence_class=peer_coordination_waitcodex_interval=10codex_max=60cli_limit/claude_limit=3example_progression_minutes=[10,20,30,60],宿主经既有 update_current_heartbeat_rrule 应用。
  3. loopx/control_plane/scheduler/automation_liveness.py::build_automation_liveness(新分支)—— 输出 automation_action=keep_active_peer_coordination_backoffkeep_active=truepause_allowed=false,并把异步恢复触发条件写进 next_trigger;位置在 monitor/quiet-wait 分支之前,避免被降级成静默等待。
  4. loopx/control_plane/todos/resume_planning.ts::deferredPlan —— 新增 itemIdentity(canonical todo id 优先,legacy 行才退回 text+index),并提供 capabilities 时对 [...deferred_resume_candidates, ...deferred_items] 去重后做 capacity 重算;deferredItems 再按原展示身份过滤,展示边界不被撑大,而 candidates 取自重算后的并集。
  5. examples/control_plane/quota-resume-gated-open-todo-smoke.py(本轮修复)—— 等优先级用例原先只把展示 lane 的 priority 改成 P1,留下无损 lane 的 P0,于是并集仍按 P0 抢占;现在两个投影一起改,fixture 重新描述"同一个 Todo"。

本轮自修复的内容(相对 PR 原 head)

原 head 的 CI 失败有三类,我逐一落到根因:

  1. CLI output budget 超限(768/798 > 750):老 base 上的既有漂移。同步 main 后,tests/control_plane/test_cli_output_budget.py 在本 head 与 main 上均通过。
  2. Frontstage build:打包资源与源码构建不一致:同为老 base 漂移;同步后本地 npm run build:chatloopx/web/chat 无任何 diff
  3. test_selection_replan_reentry.py::...[todo]:这条在 main 上也是红的(与 #4845 无关)。根因是 129bb1fe7 把 todo-bound 长链改为走 projected vision closeout 并用 test_long_chain_projected_closeout.py 替换了同型 journey,却漏改了 reentry 用例——生成的 refresh 命令里的 <path-to-evidence-linked-goal-vision-replan-contract-v0.json> 占位符无人替换,CLI 以缺文件失败。我先按既有契约补了同样的 vision packet 撰写,随后 main#4847 独立修了同一处;本轮合并该文件直接采用 main 已评审的版本,我那份等价修复自然作废,diff 里也不再含该文件。

两条路径

  • 准入路径:blocked peer + 本地 ready 候选 → 候选不丢 → disposition=wait → scheduler 输出 10/20/30/60 backoff、automation 输出 keep_active_peer_coordination_backoff → 宿主更新 RRULE,不花配额;决策身份变化时回到初始 cadence。
  • 拒绝路径:terminal/Goal stopped/quota paused 仍走 stop;结构非法执行上下文仍 contract_error 并禁用最终 replan check;等优先级不再因单 lane 旧优先级抢占。

对主干的风险

  • 唯一的默认行为变化peer_coordination_blocked 由 stop 变为可恢复 wait,host 会长期保留一个最多 60 分钟一跳、不花配额的观察者,且 unchanged_poll.final_quota_replan_check_enabled 由 false 变 true。这是本 PR 的目的,已在三处文档 + vocabulary 中披露。
  • 已验证(exact head d4fe3395btests/control_plane/test_selection_replan_reentry.py + test_cli_output_budget.py 26 passedtask-orchestration-smokequota-resume-gated-open-todo-smokeautomation-liveness-state-machine-smokesemantic-vocabulary-drift-smokepwa-bundlepresentation-surface-schema 全通过;npm run build:chat 无 diff;Ruff clean;loopx canary premerge --from-git-diffpassed、0 failures、0 manual holds(catalog 10/10、risk profile 8/8)。紧邻的前一 head(产品 diff 完全相同)上另有:npm run test:control-plane 2298→2300 用例、2280 passed / 0 failedtypecheck:control-plane 通过、10 个 scheduler/quota/settlement/vocabulary 套件 478 passed
  • 未验证:本 head 的远端 required checks 在发布评审时尚未跑完;没有真实宿主应用 RRULE;PR 正文引用的 Finance-goal base/head 回放本轮未重跑。
  • 两条非阻塞 P3:①同一 Todo 同时出现在两个 lane 时,无损 lane 的 payload 会覆盖展示 lane(刻意设计),建议把"fixture 必须同时移动同一 Todo 的每个投影"写进 deferred-lane 说明;②等待状态会让自动化长期存活,若将来需要硬性上限,应新增显式 repair 指令而不是偷偷恢复 stop。

我的整体评价

这是一次方向正确的收口:把"peer 阻塞"从自相矛盾的 stop 改成复用既有 typed backoff 的可恢复 wait,同时修掉能力重算时丢候选的真实缺陷;实现上删掉专用 stop 分支而不是新增 reducer,scheduler 与 automation 两个投影现在说同一句话,文档与 vocabulary 同步更新,符合仓库"状态机与 cadence 权威留在 TS typed 边界、优先退休特殊路径"的约束。

本轮我做的三件事都只为让验收证据成立:同步 main 消掉两类老 base 漂移、合并 main 对 reentry 契约的独立修复(采用其已评审版本)、把 resume-gated smoke 的 fixture 从"同一 Todo 两个优先级"改回一致。没有放宽任何断言。

结论:APPROVE(本 head 无阻塞问题)。建议合并后确认该 head 的远端 required checks 收口。


English verdict: APPROVE - Reviewed exact head d4fe339 (PR head 216fc90 synced with main@154f71a6c plus one review-round fixture repair). The PR fixes both halves of #4843: resume planning re-evaluates the lossless candidate lane together with the bounded display lane (union + canonical-todo-id dedup, display bound preserved, explicit candidate payload wins), and peer_coordination_blocked is reclassified from a stop to a recoverable wait that reuses the existing TypeScript-owned stateful backoff (backoff_until_reassigned, cadence class peer_coordination_wait, 10/20/30/60) with automation liveness keep_active_peer_coordination_backoff and a no-spend policy, disclosed in three docs and the effective-action vocabulary. The pre-sync CI failures were a stale base (CLI output budget, packaged chat assets) and a stale reentry contract already repaired on main by #4847, whose reviewed version this merge takes verbatim; the only surviving review-round edit is a 9-line fixture repair in examples/control_plane/quota-resume-gated-open-todo-smoke.py so the equal-priority case moves priority in both projections of the same Todo. Validation: 26 passed for the reentry and budget suites at this head, 478 focused Python tests and the full TS control-plane suite (2280 pass / 0 fail) on the identical product diff immediately before the last main sync, six smokes and the packaged-asset build check green, ruff clean, and the exact-diff canary premerge gate passed with 0 failures and 0 manual holds. Two non-blocking P3 notes (lossless-lane payload ownership should be documented for fixtures; the wait keeps an automation alive indefinitely by design) plus residual risk: remote CI for this head was still running, no live host applied a real RRULE, and the Finance-goal replay was not re-executed.

…backoff

Signed-off-by: LoopX Agent <agent@loopx.local>
…on reentry

129bb1f moved a todo-bound long chain onto projected vision closeout and
replaced the analogous settlement journey, but left
test_selection_replan_reentry.py substituting only the progress identifiers.
The generated refresh then carried an unfilled
<path-to-evidence-linked-goal-vision-replan-contract-v0.json> and the CLI
failed with a missing-file error, so main has been red on this test.

Author the evidence-linked vision packet for that branch, mirroring
tests/control_plane/test_long_chain_projected_closeout.py, so the test
proves the deferred selection exposes a runnable recovery instead of
asserting the retired progress-only guidance.

Signed-off-by: LoopX Agent <agent@loopx.local>
The bounded display lane and the lossless candidate lane project the same
canonical Todo, and resume planning now prefers the explicit candidate lane
when both carry that id. The smoke moved only the compact lane, so the fixture
described one Todo at two different priorities and the equal-priority case
still preempted on the stale P0. Move the priority in both projections so the
fixture models one real Todo and the equal-priority expectation holds.

Signed-off-by: LoopX Agent <agent@loopx.local>
…backoff

Conflict was tests/control_plane/test_selection_replan_reentry.py: main's
#4847 fixed the same stale contract independently (authoring the
evidence-linked vision packet for the todo binding). Take main's reviewed
version verbatim; this branch's equivalent repair is superseded.

Signed-off-by: LoopX Agent <agent@loopx.local>
@huangruiteng
huangruiteng force-pushed the codex/fix-ready-work-backoff branch from d4fe339 to 588b318 Compare September 21, 2026 09:53

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Exact head: 588b318c02b4d3cd55796f887f308e64007a2fba(PR head 216fc90cc + main@154f71a6c 同步 + 本轮一处 fixture 修复;head 因 DCO 重写后刷新,tree 与 d4fe3395b 完全一致)

动机

#4843 里其实是两个互相掩盖的缺陷。其一:当宿主提供 runtime capabilities 时,resume planning 只用有界展示 lanedeferred_items)重算候选,把只存在于无损候选 lanedeferred_resume_candidates)里的 ready 行丢掉,于是"有可做的工作"被投影成 peer_coordination_blocked,工作被搁置。其二:同一状态下 scheduler 与 automation 自相矛盾——keep_active=true 却给出 host_action=pause_or_delete_current_heartbeatlocal_scheduler=stop,等于一边说"保持活跃",一边让宿主停掉唯一的观察者。

PR 的立场很清楚:peer 阻塞的恢复输入(peer capability/readiness、协调配置、重新分配、本地 frontier)都可能在本 Agent 之外异步变化,所以它应当是可恢复的 wait,而不是 stop;真正的 stop 只保留 Goal stopped、quota paused、已验证的 terminal no-follow-up。

改动思路

  • 判定归位SchedulerDisposition.PEER_COORDINATION_STOPPEER_COORDINATION_WAIT,并把 scheduler_hint.py 里那段专用的 peer stop 分支删掉,改为复用 agent-scope wait 已经在用的 typed stateful backoff(backoff_until_reassigned + cadence_class=peer_coordination_wait + 10/20/30/60),而不是再加一个 leaf reducer 或 bridge。
  • 两个投影对齐automation_liveness 增加显式分支,输出 keep_active_peer_coordination_backoff,与 scheduler 的"保持无花费观察者"一致;spend_policy 明确 no quota spend。
  • 候选不丢resume_planning.ts 用 canonical Todo id 做 identity 去重,并在提供 capabilities 时对两个 lane 的并集做 capacity 重算,再按原展示身份把 deferred_items 收敛回有界展示窗口;并集里显式候选 lane 放前面,因此同一 Todo 以无损 payload 为准。
  • 披露到位docs/quota-allocation.mddocs/product/core-control-plane/state-machine.md(cadence 表新增 peer_coordination_wait)、docs/integrations/codex-subagent-orchestration.mdloopx/semantics/vocabulary_v0.json 的 effective-action 描述同步更新;没有新增 schema、CLI flag 或持久化字段。

具体改动

关键代码讲解

  1. loopx/control_plane/scheduler/arbitration.py::SchedulerDisposition.PEER_COORDINATION_WAIT —— 只改分类,不改 payload 语义;terminal/terminal_no_followup 分支仍在其之前返回,所以停止语义不会被 wait 吞掉。
  2. loopx/control_plane/scheduler/scheduler_hint.py::build_scheduler_hint(peer 分支)—— 从"专用 stop hint"改为共用 builder:action=backoff_until_reassignedcadence_class=peer_coordination_waitcodex_interval=10codex_max=60cli_limit/claude_limit=3example_progression_minutes=[10,20,30,60],宿主经既有 update_current_heartbeat_rrule 应用。
  3. loopx/control_plane/scheduler/automation_liveness.py::build_automation_liveness(新分支)—— 输出 automation_action=keep_active_peer_coordination_backoffkeep_active=truepause_allowed=false,并把异步恢复触发条件写进 next_trigger;位置在 monitor/quiet-wait 分支之前,避免被降级成静默等待。
  4. loopx/control_plane/todos/resume_planning.ts::deferredPlan —— 新增 itemIdentity(canonical todo id 优先,legacy 行才退回 text+index),并提供 capabilities 时对 [...deferred_resume_candidates, ...deferred_items] 去重后做 capacity 重算;deferredItems 再按原展示身份过滤,展示边界不被撑大,而 candidates 取自重算后的并集。
  5. examples/control_plane/quota-resume-gated-open-todo-smoke.py(本轮修复)—— 等优先级用例原先只把展示 lane 的 priority 改成 P1,留下无损 lane 的 P0,于是并集仍按 P0 抢占;现在两个投影一起改,fixture 重新描述"同一个 Todo"。

本轮自修复的内容(相对 PR 原 head)

原 head 的 CI 失败有三类,我逐一落到根因:

  1. CLI output budget 超限(768/798 > 750):老 base 上的既有漂移。同步 main 后,tests/control_plane/test_cli_output_budget.py 在本 head 与 main 上均通过。

  2. Frontstage build:打包资源与源码构建不一致:同为老 base 漂移;同步后本地 npm run build:chatloopx/web/chat 无任何 diff

  3. test_selection_replan_reentry.py::...[todo]:这条在 main 上也是红的(与 #4845 无关)。根因是 129bb1fe7 把 todo-bound 长链改为走 projected vision closeout 并用 test_long_chain_projected_closeout.py 替换了同型 journey,却漏改了 reentry 用例——生成的 refresh 命令里的 <path-to-evidence-linked-goal-vision-replan-contract-v0.json> 占位符无人替换,CLI 以缺文件失败。我先按既有契约补了同样的 vision packet 撰写,随后 main#4847 独立修了同一处;本轮合并该文件直接采用 main 已评审的版本,我那份等价修复自然作废,diff 里也不再含该文件。

  4. Sign-off(DCO)红灯:本轮我先用 git merge --no-edit 建的合并提交没带 Signed-off-by,被 DCO 门禁拦下。已重写该分支历史,让 origin/main..HEAD每个提交都带合法 trailer(tree 未变,逐字节等于已验证的 d4fe3395b),新 head 为 588b318c0

两条路径

  • 准入路径:blocked peer + 本地 ready 候选 → 候选不丢 → disposition=wait → scheduler 输出 10/20/30/60 backoff、automation 输出 keep_active_peer_coordination_backoff → 宿主更新 RRULE,不花配额;决策身份变化时回到初始 cadence。
  • 拒绝路径:terminal/Goal stopped/quota paused 仍走 stop;结构非法执行上下文仍 contract_error 并禁用最终 replan check;等优先级不再因单 lane 旧优先级抢占。

对主干的风险

  • 唯一的默认行为变化peer_coordination_blocked 由 stop 变为可恢复 wait,host 会长期保留一个最多 60 分钟一跳、不花配额的观察者,且 unchanged_poll.final_quota_replan_check_enabled 由 false 变 true。这是本 PR 的目的,已在三处文档 + vocabulary 中披露。
  • 已验证(exact head 588b318c0,tree 与 d4fe3395b 相同)tests/control_plane/test_selection_replan_reentry.py + test_cli_output_budget.py 26 passedtask-orchestration-smokequota-resume-gated-open-todo-smokeautomation-liveness-state-machine-smokesemantic-vocabulary-drift-smokepwa-bundlepresentation-surface-schema 全通过;npm run build:chat 无 diff;Ruff clean;loopx canary premerge --from-git-diffpassed、0 failures、0 manual holds(catalog 10/10、risk profile 8/8);origin/main..HEAD 每个提交的 DCO trailer 已本地逐条核验。紧邻的前一 head(产品 diff 完全相同)上另有:npm run test:control-plane 2300 用例、2280 passed / 0 failedtypecheck:control-plane 通过、10 个 scheduler/quota/settlement/vocabulary 套件 478 passed
  • 未验证:本 head 的远端 required checks 在发布评审时尚未跑完;没有真实宿主应用 RRULE;PR 正文引用的 Finance-goal base/head 回放本轮未重跑。
  • 两条非阻塞 P3:①同一 Todo 同时出现在两个 lane 时,无损 lane 的 payload 会覆盖展示 lane(刻意设计),建议把"fixture 必须同时移动同一 Todo 的每个投影"写进 deferred-lane 说明;②等待状态会让自动化长期存活,若将来需要硬性上限,应新增显式 repair 指令而不是偷偷恢复 stop。

我的整体评价

这是一次方向正确的收口:把"peer 阻塞"从自相矛盾的 stop 改成复用既有 typed backoff 的可恢复 wait,同时修掉能力重算时丢候选的真实缺陷;实现上删掉专用 stop 分支而不是新增 reducer,scheduler 与 automation 两个投影现在说同一句话,文档与 vocabulary 同步更新,符合仓库"状态机与 cadence 权威留在 TS typed 边界、优先退休特殊路径"的约束。

本轮我做的三件事都只为让验收证据成立:同步 main 消掉两类老 base 漂移、合并 main 对 reentry 契约的独立修复(采用其已评审版本)、把 resume-gated smoke 的 fixture 从"同一 Todo 两个优先级"改回一致。没有放宽任何断言。

结论:APPROVE(本 head 无阻塞问题)。建议合并后确认该 head 的远端 required checks 收口。


English verdict: APPROVE - Reviewed exact head 588b318 (PR head 216fc90 synced with main@154f71a6c plus one review-round fixture repair). The PR fixes both halves of #4843: resume planning re-evaluates the lossless candidate lane together with the bounded display lane (union + canonical-todo-id dedup, display bound preserved, explicit candidate payload wins), and peer_coordination_blocked is reclassified from a stop to a recoverable wait that reuses the existing TypeScript-owned stateful backoff (backoff_until_reassigned, cadence class peer_coordination_wait, 10/20/30/60) with automation liveness keep_active_peer_coordination_backoff and a no-spend policy, disclosed in three docs and the effective-action vocabulary. The pre-sync CI failures were a stale base (CLI output budget, packaged chat assets) and a stale reentry contract already repaired on main by #4847, whose reviewed version this merge takes verbatim; the only surviving review-round edit is a 9-line fixture repair in examples/control_plane/quota-resume-gated-open-todo-smoke.py so the equal-priority case moves priority in both projections of the same Todo. Validation: 26 passed for the reentry and budget suites at this head, 478 focused Python tests and the full TS control-plane suite (2280 pass / 0 fail) on the identical product diff immediately before the last main sync, six smokes and the packaged-asset build check green, ruff clean, and the exact-diff canary premerge gate passed with 0 failures and 0 manual holds. Two non-blocking P3 notes (lossless-lane payload ownership should be documented for fixtures; the wait keeps an automation alive indefinitely by design) plus residual risk: remote CI for this head was still running, no live host applied a real RRULE, and the Finance-goal replay was not re-executed.

@huangruiteng
huangruiteng merged commit 59e1111 into main Sep 21, 2026
6 checks passed
@huangruiteng
huangruiteng deleted the codex/fix-ready-work-backoff branch September 21, 2026 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ready deferred work can be hidden and peer waits can stop heartbeats

1 participant