feat(handoff): lossless ordered shards for over-budget project-agent handoff - #4444
fengyin-solo wants to merge 2 commits into
Conversation
Project-agent handoffs have a fixed 16 line / 1800 character interface budget. The old overflow pass first compacted the command block and then deleted whole sections by fixed prefixes, returning an over-budget text when deletion was insufficient: receivers lost content silently and could not tell whether more existed. Replace the lossy section drop with ordered, independently verifiable shards in control_plane/handoff/handoff_fragments.py: - split_handoff_text returns the input verbatim (no envelope) when it fits; over-budget text becomes shards, and shard 0 keeps the existing project_agent_handoff position and field semantics; - each shard carries an envelope with a content-derived stable set id, i/total sequence, per-shard payload checksum, previous-shard hash chain, and full-content SHA-256; - reassemble_handoff_shards/restore_handoff_text validate every payload, set consistency, the hash chain, and full digest, failing explicitly on missing, out-of-order, duplicate/conflicting, foreign- set, or tampered shards; - HandoffShardCollector imports shards idempotently (same set/index/ checksum is a no-op), so regeneration and re-import never duplicate fragments; - fenced command blocks are never split open (strip-only close/reopen markers balance every shard), and over-long single lines wrap at safe whitespace boundaries with hard-cut fallback via continuation markers, restoring byte-for-byte; - build_review_packet exposes project_agent_handoff_fragments plus a compact handoff_fragment_manifest only when fragmented; full packet and handoff-only markdown render every shard, and handoff-only JSON passes the fragments through. Within-budget output is byte-identical. Tests cover split/restore, missing/out-of-order/tamper/digest errors, idempotent regeneration and import, over-long lines and fence splitting, a budget matrix, packet/handoff-only integration, and the within-budget compatibility shape. Signed-off-by: fengyin-solo <292015025+fengyin-solo@users.noreply.github.com>
Describe the lossless overflow protocol: shard 0 keeps the existing project_agent_handoff semantics, continuation shards use a stable content-derived set id with index/total, per-shard checksum, hash chain, and full digest, and receivers fail explicitly on missing, out-of-order, duplicate, or tampered shards. Document fence close/reopen and line continuation handling, idempotent re-import, and byte-identical output within the 16 line / 1800 character budget. Signed-off-by: fengyin-solo <292015025+fengyin-solo@users.noreply.github.com>
huangruiteng
left a comment
There was a problem hiding this comment.
动机
超预算 handoff 之前是有损的:fit_project_agent_handoff_budget 会先压缩最后一段 bash 块,然后按前缀逐组丢掉 Agent 待办候选、材料上下文:、交付观测:、交付合同:,删完仍然超预算就照样返回那份超预算文本。把交付合同/交付观测丢掉,项目 Agent 就会按不完整的契约干活,所以"不要再丢内容"这个方向我认同。本 head 改成:整段归一化后(只做无损的 bash 块压缩)切成带信封的有序分片,project_agent_handoff 放第 0 片,其余进 project_agent_handoff_fragments。方向没问题,但改的是对接收端可见的输出契约,有两处必须先收口。
改动思路
新增 loopx/control_plane/handoff/handoff_fragments.py:信封行 <!--loopx-handoff v=1 id=… i=… n=… c=… p=… d=…--> 承载内容派生的 set id、序号/总数、单片校验和、前片哈希链与全文 digest;packer 保证围栏不被切开、超长行用续行标记折行;读取侧提供 parse_handoff_shard/is_handoff_shard_text/reassemble_handoff_shards/restore_handoff_text/HandoffShardCollector/extract_handoff_shards/build_handoff_shard_manifest,并在编码末尾用自校验保证逐字节可逆。review_packet.py 里旧的丢段函数被删除,改由 normalize_project_agent_handoff_text + split_handoff_text 承担;cli_commands/status.py 让 handoff-only markdown 打印全部分片,JSON 额外暴露 fragments 与 manifest。
具体改动
loopx/control_plane/handoff/handoff_fragments.py(新,736 行):信封、packer/折行、解析与校验、collector、extractor、manifest。loopx/review_packet.py(+92/-26):删除fit_project_agent_handoff_budget,新增normalize_project_agent_handoff_text、prepare_project_agent_handoff_shards、render_handoff_only_text;packet 侧agent_text = handoff_shards[0](第 711-712 行)。loopx/cli_commands/status.py(+21):handoff-only JSON 透传project_agent_handoff_fragments/handoff_fragment_manifest(143-145),markdown 走render_handoff_only_text(898-905)。docs/status-data-contract.md(+27/-2):登记分片契约与逐字节可逆承诺。tests/test_handoff_fragments.py(+560,新文件):32 个测试覆盖分片、缺失/乱序/篡改/重复、围栏与续行。
关键代码讲解
split_handoff_text(handoff_fragments.py:285):未超预算时原样返回[text](无信封),超预算才分片;每片生成后再核对len(shard.split("\n")) <= 16、len(shard) <= 1800,最后reassemble_handoff_shards(shards) != text就抛错——编码器自校验,这点做得扎实。review_packet.py:711-712:handoff_shards = split_handoff_text(prepared_agent_text)、agent_text = handoff_shards[0]。这一行决定了project_agent_handoff/handoff_text的语义在超预算时从"完整文本"变成"第 0 片"。review_packet_handoff_only_payload(status.py:128-129,143-145):handoff_text复制的就是project_agent_handoff,即第 0 片。
对主干的风险
阻塞项 1(P1,接收端契约没跟上,且校验入口不可达)。我用真实 CLI 做了反事实:live loopx-meta 的 handoff 是 14 行/1370 字符(未超预算,--handoff-only 输出与 head 之外无差异);把它加上 必读、成员、交付合同 三行变成 17 行/1541 字符后,markdown 输出分成 2 片(shard 0 = 15 行/1569 字符,shard 1 = 4 行/272 字符),而可执行的 ```bash 命令落在第 2/2 片;JSON 里 handoff_text/`project_agent_handoff` 只等于 shard 0。但 `skills/loopx-project/SKILL.md:700` 仍写着"JSON 返回带 `handoff_text` 的 minimized handoff",`:710` 仍把 `给项目 Agent` 说成 "the executable handoff context"——本 PR 只更新了 `docs/status-data-contract.md`,没更新这个接收端 skill。全仓 `rg` 的结果是:`reassemble_handoff_shards`/`restore_handoff_text`/`HandoffShardCollector`/`extract_handoff_shards` 除了本模块与单测没有任何调用方,也没有 CLI 入口,所以转发文案里承诺的"收齐全部 N 片并按序号校验通过后再执行;缺片、乱序或内容改动都会明确报错"在接收端其实不可达:我把只含 shard 0 的 `handoff_text` 交给 `restore_handoff_text` 会抛 `code=missing`,可接收 Agent 没有任何 shipped 途径走到这一步,若它直接按字段执行,则既没有命令、也没有报错。最小修复二选一:(a) 补一个接收端可用的 verify+restore 入口(例如文档化的 `loopx review-packet --restore-handoff`)并写进 skill;(b) 分片时把 `handoff_text`/`project_agent_handoff` 显式标成"不可单独执行",并同步改 `skills/loopx-project/SKILL.md` 与转发文案,让按 skill 行动的 Agent 不可能只执行第 0 片。
阻塞项 2(P1,改动落在 CLI 契约上却没有真实入口级验证)。examples/control_plane/review-packet-cli-smoke.py 本 PR 未改动,它的 dense fixture 在 head 上仍是 13 行/1319 字符(未分片),所以"markdown 打印多片""JSON 新增 project_agent_handoff_fragments + handoff_fragment_manifest"这两条被改动的契约在 CLI/契约层零覆盖;新覆盖全在 tests/test_handoff_fragments.py(560 行、合成超行文本)。我实测:pytest tests/test_handoff_fragments.py → 38 passed;review-packet-cli-smoke.py、runtime-handoff-status-read-path-smoke.py、platform-migration-material-registry-smoke.py 均通过——正因为它们都不触发分片。本仓库对 status/packet 这类表面的改动要求比单元测试更强的证据。最小修复:在 public CLI smoke 补一个超预算 case(断言 markdown 的分片段、JSON 的 fragments+manifest、以及一次可校验的重组),并把 loopx canary premerge --from-git-diff 的结果贴到 PR 评论。
P3(体量与触发条件):736 行新模块 + 560 行单测,而触发条件(16 行/1800 字符)在仓库现有 fixture 与 live goal 上都没有被观察到(loopx-meta 14 行、dense fixture 13 行,我只在人为加三行后才触发)。同时旧的丢段回落被整体删除,packet 路径不再有任何更简单的兜底。建议把"验证的一半"(collector/extractor/manifest/transport markers)留到真有接收端调用点再上,或保留现有单文本行为作为第一层、只在仍超预算时才分片;已上线的部分请让每个 helper 都有活的调用方。
其余非阻塞观察:ENVELOPE_CHAR_RESERVE=200 相对实际信封长度(我构造的两片分别是 142/157 字符)是保守的,我的 hostile 用例没有出现预算越界;分片标题行只出现在后续片上并写作「交接分片 2/2」,第 0 片没有对应标题,读者容易误判第 0 片是完整文本;splitter 对以信封前缀/续行标记开头的正文是 fail-closed,这一点只写在模块 docstring 里,没有写进接收端文档。
我的整体评价
REQUEST_CHANGES。无损 relay 的方向我认同,实现本身也自洽:信封把 set id 绑到内容 digest,每片校验和 + 前片哈希链 + 全文 digest 齐全,围栏不被切开、超长行可逆折行,编码末尾还有逐字节自校验,未超预算路径与既有 smoke 全绿。但这次改的是 review-packet/handoff 对接收端可见的输出契约:超预算时 project_agent_handoff/handoff_text 不再是可直接执行的完整文本(命令跑到最后一片),而接收端既没有 skill/文案更新、也没有任何 shipped 的校验或重组入口;同时被改动的 CLI 契约没有任何真实入口级覆盖。收口建议很小:补接收端入口或显式声明不可单独执行 + 更新 skill,补一个 CLI 超预算 case 并跑 canary premerge;如果想更保守,也可以按上面 P3 收缩成"单文本优先、仅必要时分片"。改完后请让我按新的 exact head 再做一次复审。
English verdict: REQUEST_CHANGES at 7a04a3e. Keeping over-budget handoffs lossless is the right goal, and the relay implementation is internally consistent (content-derived set id, per-shard checksum, previous-shard hash chain, full-content digest, fence-safe packing, byte-exact encoder self-check; 38 focused tests pass and the in-budget path is unchanged in a live CLI run). The problem is the receiving contract: for an over-budget handoff, project_agent_handoff/handoff_text become shard 0 only. In my probe of a realistic 17-line handoff the executable ```bash command lands in shard 2/2, so the documented minimized field is not executable, while skills/loopx-project/SKILL.md still presents that field and the 给项目 Agent section as the executable handoff and was not updated by this PR. The promised explicit shard errors are also unreachable from the receiver surface: reassemble_handoff_shards/restore_handoff_text/HandoffShardCollector/extract_handoff_shards have no caller outside this module and its tests, and no CLI entry point exists. Secondly, the changed CLI contract has no entry-point coverage: review-packet-cli-smoke.py was not updated and its dense fixture stays in budget (13 lines / 1319 chars), so the new markdown shard sections and the new JSON fragment fields are covered only by unit tests over synthetic text. Please add a receiver-reachable verify+restore surface (or mark the field non-executable and update the skill), add an over-budget CLI smoke case, and post loopx canary premerge results, then I will re-review the new exact head.
RFC 对齐补充评审评审对象: 本次按请求补充 RFC 定位与交付顺序;已有完整评审仍对应同一 head,正式 REQUEST_CHANGES 结论保持不变。以下不重复发起一次相同提交的全面审计,也不把未来 RFC 验收项全部追加为本 PR 阻塞项。 动机保留超预算交接中的约束、材料和交付要求,符合强能力管家与语义交接 RFC §5.4:投影应披露遗漏,不能静默删除用户约束。不过该节同时要求小而适配任务的投影,以及可实际读取的完整材料。逐字重组解决文本传输完整性,只能作为语义交接的一个基础环节。 改动思路建议将本 PR 定位为 现有 review-packet 的无损传输与兼容性修复。新增 Python codec 可以留在现有 handoff 边界;本次没有接管 Todo、claim、lease 或 request 状态机,没必要为了目录名或未来 M2 而增加 Python↔TypeScript 往返。 管家 RFC §11/§11.1允许独立的格式/投递修复先于通用协作重构交付。因而本 PR 不必等待 M2 双消费者、M3 自动回传或共享存储提升;也不应以分片功能完成来宣告这些里程碑已完成。 具体改动本次对照了现有评审、PR 描述及相关源码:
按 RFC 节奏建议这样收口:
M2 的身份与回执边界来自§5.6:内容哈希不能合并独立请求,传输、评估、执行和答案送达是独立事实。未来如迁移这些状态规则,再执行TS 迁移 RFC §5的 owner/删除旧规则/桥接成本证明;纯传输 codec 不需要伪装成一次状态机迁移。 对主干的风险当前主要风险仍是完整文本字段变成首片后,旧接收方式可能遗漏约束或命令,以及真实 CLI 溢出路径的证据不足。这里的“收齐再执行”需要落实到接收路径;仅写提示不能声称已有机器强制保证。 还有两项非阻塞的演进约束:
结合刚合并的语义词汇 M0,下一修订更新到当前基线时,应检查新增协议常量是否使生成 inventory 过期,并运行 我的整体评价保留 REQUEST_CHANGES,支持修复接收闭环后作为独立传输改进交付。 先完成既有接收端兼容与 CLI 验证,再把请求身份/语义评估、自动回传分别放进 M2/M3。未来重构检查的具体边界是 codec 与协作状态机:当前建议收敛到真实调用,避免增加第二个请求或权威状态 owner。 验证说明:本次是同一 exact head 的 RFC 补充,对照了远端讨论、上述版本化 RFC、字段投影及调用点;没有重新运行全面测试,没有生成新的全面审计或合并资格结论。先前测试结果见原评审,不能当作下一修订的验证。按本 Goal 配置,本次未查询、轮询或等待 CI;接收端与本地验证缺口仍需补齐。 English verdict: REQUEST_CHANGES remains at 7a04a3e. This RFC alignment supplement preserves the existing exact-head review: complete receiver compatibility and real over-budget CLI coverage first. Keep sharding a bounded transport codec; reserve semantic request identity/assessment for M2 and durable result return for M3. RFCs and source call sites were inspected; tests were not rerun and CI was not consulted. |
Change request:把交接上下文从 Review Packet 中分离,补齐可用的接收路径对应 head: 这是维护者针对长期 handoff 方向提出的具体修订要求,补充已有 REQUEST_CHANGES,并细化上一条 RFC 意见。本次希望包含一次有界的职责重构,而不止增加分片恢复命令。 这是明确的设计范围调整,不是声称在同一 head 上又复现了一个新的运行时缺陷。 动机当前 长期 handoff 要支持有上下文的工作委派、接收方重新评估、会话替换和结果回传。因此需要调整依赖方向:交接上下文有自己的组装边界,Review Packet 消费它来生成人工可读投影,传输层再按渠道需要编码。 依据:handoff RFC §5.2–5.6。该 RFC 已要求区分应用、协作语义和运行时/渠道适配,现有模块位置是迁移输入。 改动思路建议本次交付路径如下;方框表示职责,不要求照此创建同名模块或公共 schema: 这里的“共同”必须由真实的 full packet 与 handoff-only 路径消费。不要仅搬一个函数、保留两份事实推导,也不要为未来消费者建立空框架。接收端恢复成功只证明内容完整,不代表已接受工作、获得 claim/lease 或可直接执行其中的命令。 具体改动CR1:提取上下文组装职责,保留 Review Packet 的展示职责请先列出
完成标准: 状态/上下文事实只有一个组装来源,两个现有输出路径均消费它;旧的重复组装逻辑在本 PR 删除,调用关系与职责可以从源码直接看清。 CR2:分片属于传输;修正完整文本字段的语义当前
完成标准: 完整交接、单片和恢复结果在契约上不混淆;未收齐内容的兼容消费者没有“成功收到完整交接”的假象。 CR3:交付接收方实际能用的校验/恢复入口请提供一个已文档化、可以由接收 Agent 调用的真实入口,并在 managed skill 中写出可运行步骤。它应属于 handoff 的内容接收职责;不要仅因当前生产者是 具体命令拼写可以沿用仓库注册模式;若选用现有
完成标准: 一个接收方仅凭 shipped CLI/skill 即可恢复合法交接,识别不完整输入;无需临时写 Python 去导入内部函数。 CR4:补齐 RFC replacement map 与公开操作说明请更新 handoff RFC §5.9 的替换表及中英文镜像,明确登记
同时修正 对主干的风险CR5:按真实消费路径补验证新增测试应先从下面这些语义要求写出预期,再测试实现。不要把实现生成的内容反过来当作唯一 oracle。
复用并扩展 下一 head 请运行相关 focused tests、现有 packet/CLI 回归,以及 我的整体评价REQUEST_CHANGES 保持。建议把“上下文组装职责分离 + 明确的兼容表示 + 接收端闭环 + 聚焦验收”作为一个可独立交付的修订。 它共享同一个修改原因,属于本 PR 合适的配套重构。 本次不要求实现通用 M2 状态机、M3 自动回传、跨主机持久化、历史召回、新 artifact 服务或整体 TypeScript 迁移。纯文本/投影逻辑可以保留 Python;未来真正迁移协作状态决策时,再按 TS 迁移 RFC证明单一 owner、旧规则删除与桥接成本。 也不要求现在删除 请在下一修订说明中逐项回复 CR1–CR5,给出实现位置与验证结果;实现选择可以不同,只要满足上述可观察契约。这条评论是具体修订要求,不是新一轮测试通过或合并资格证明。 English verdict: REQUEST_CHANGES remains at 7a04a3e. Please extract the existing handoff-context assembly from Review Packet presentation, preserve legacy compatibility without exposing shard 0 as a complete handoff, ship a receiver-accessible verify/restore path, update the RFC replacement map and managed skill, and validate real producer-to-receiver CLI flows. Keep this a bounded refactor; generic M2/M3 lifecycle, ownership transfer, and storage migration remain separately qualified work. This comment specifies the requested revision; it does not claim new test execution or consult CI. |
Summary
Agent 待办候选,材料上下文,交付观测,交付合同), and returned a still-over-budget text when deletion was insufficient: receivers lost content silently and could not tell whether more existed.loopx/control_plane/handoff/handoff_fragments.py:split_handoff_textreturns the input verbatim with no envelope when it fits; over-budget text becomes shards, and shard 0 keeps the existingproject_agent_handoffposition and field semantics.<!--loopx-handoff ... -->envelope carrying a content-derived stable set id,i/nsequence, a per-shard payload checksum, the previous-shard hash chain, and the full-content SHA-256.reassemble_handoff_shards/restore_handoff_textvalidate every payload, set consistency, sequence/hash chain, and full digest. Missing shard, out-of-order delivery, duplicate/conflicting import, foreign-set shard, or altered content fail with explicit typed error codes (missing,out_of_order,duplicate,set_mismatch,integrity,digest, ...).HandoffShardCollectoraccumulates imports idempotently: same set/index/checksum is a no-op, so regenerating the same handoff yields byte-identical shards and re-import never duplicates fragments.build_review_packetexposesproject_agent_handoff_fragmentsplus a compacthandoff_fragment_manifestonly when fragmented; the full packet and handoff-only markdown render every shard, and handoff-only JSON passes both fields through. When the handoff fits the budget, text, packet, and JSON payload are byte-identical to before (no envelope, no new keys).docs/status-data-contract.md.Issue Or Task
Validation
9f16df079runtime commit /7a04a3e70docs commit, rebased ontoupstream/main(503991dd2)unitpassedpytest tests/test_handoff_fragments.py— 38 tests: split/restore, missing/out-of-order/tamper/chain/digest errors, stable regeneration, idempotent import, over-long spaced/unbreakable/unicode lines, fenced block splitting across shards, budget matrix, packet and handoff-only integration, within-budget compatibility.regression_paritypassedpytest tests/control_plane/test_cli_output_budget.py tests/test_cli_entrypoint.py tests/test_cli_argument_diagnostics.py tests/test_manager_context_handoff.py tests/architecture/test_control_plane_import_boundaries.pytogether with the new tests — 196 passed; existing review-packet fixtures (including the dense over-line-count case that uses command-block compaction) produce identical single-text handoffs.regression_paritypassedpython3 examples/control_plane/review-packet-cli-smoke.pyandreview-packet-handoff-context-smoke.py— both pass; the smoke's compactness/forbidden-marker assertions still hold for shard 0, and CLI review-packet markdown/JSON/handoff-only paths carry fragmented handoffs.staticpassedruff checkclean on all changed/new files;loopx checkpublic-boundary scan on all changed paths (via canary).real_entrypointpassedloopx canary premerge --from-git-diff: catalog canaries 9/9, risk-profile smokes 8/8, public boundary 1/1, 18/18 total, 0 failures. Includes hot-path interface budget, CLI output budget regression, and status/quota/review-packet parity smokes.build_review_packet/CLI smoke boundaries with synthetic oversized commands and within-budget fixtures. No persistence, quota, scheduler, frontend, or benchmark surface changes; no real backend gate applies. Over-budget scenarios only arise from unusually large inputs (e.g. an oversized approvedagent_command), which existing product fixtures never hit.Frontend / Visual Evidence
Type Of Change
LoopX Area
Technical Direction
Core control-plane hardening
Long-horizon benchmark evidence
Operator surface and IM integration
Shared Goal Authority and cross-host coordination
Architecture and research incubator
Target base branch:
main(huangruiteng/loopx)Direction tracker or promotion unit: N/A
Shared-authority RFC fixture impact
N/A — no TypeScript control-plane or shared Goal Authority surface is changed.
Boundary Checklist
.loopx/,.codex/goals/, and liveACTIVE_GOAL_STATE.md).none.Signed-off-bytrailer (git commit -s).