test(ci): stabilize state-lock mutation oracle - #4863
Conversation
Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
huangruiteng
left a comment
There was a problem hiding this comment.
审阅 exact head:6dcb8c763021750011d9b96c329f85e8571fb126
动机
这个 PR 修复的是 Stage 2C mutation suite 的证据可靠性,而不是生产锁逻辑。fence_unshared_state_lock 故意把 withFileMutationLock(statePath, ...) 改成另一个锁路径;旧映射却通过 Python 启动 Node 子进程后,在 200ms 观察窗里判断 fence 是否提前完成。这样观察窗同时包含进程启动成本,机器稍慢时可能把真正的 lock-path mutant 误报为 survivor。该问题会让 required mutation gate 产生噪声,维护者既不能放心信任失败,也不能安全忽略它。
最小修复就是本 PR 的三行改动:不增加 timeout、不修改生产路径,也不新造测试,而是复用已经直接覆盖同一状态锁不变量的 TypeScript 测试。
改动思路
mutants.py 仍负责在隔离副本中先跑 unchanged control,再施加同一个 fence_unshared_state_lock mutation,最后只接受 assertion failure 作为 killed。变化仅是 oracle:从跨进程 pytest selector 改为 tests/control_plane_ts/shadow_native_writer_boundary.test.ts,并用锚定的 test-name pattern 精确选择 state writer 场景。
正向路径中,测试持有 canonical statePath lock,真实 engageLegacyCoordinationWriterFence 必须等待;mutation 路径把 fence 改到 statePath + ".mutant-unshared",于是 fence 会在 canonical lock 仍被持有时提前完成,命中 completed === false 的断言。锁身份与断言在同一个 Node 进程里,消除了与被测不变量无关的子进程启动时序。
具体改动
本 PR 仅修改 examples/shared-goal-authority-e2e/mutants.py:2 行新增、1 行删除,没有生产代码、协议、默认行为或持久化状态变化。
关键代码讲解
Case("fence_unshared_state_lock", ...)保留原 deliberate replacement,因此被审的 production invariant 没有偷换。tests/control_plane_ts/shadow_native_writer_boundary.test.ts是已有 closest-owner oracle,直接调用真实 TypeScript fence 和真实文件锁。^fence engagement waits for an existing state writer before publication$精确绑定state分支,不会误跑相邻的 todo/lease 用例;runner 还会先要求 unchanged control 出现pass 1,空匹配或坏映射会 fail closed。
对主干的风险
主要风险是新 pattern 漂移、选错测试或只让普通测试通过却不能杀死 mutation。实测覆盖了这三层:
- targeted mutation:
control_exit=0、mutant_exit=1、killed_by_assertion=true; - direct Node control:1 passed、0 failed;
tests/test_python_ci_workflow.py:44 passed;Ruff 与git diff --check通过;- standard premerge gate:6 个 selected checks 全部通过,0 failures、0 manual holds,public boundary clean。
Goal 配置为 wait_for_ci=false,因此没有把 GitHub CI 当作审阅证据,也没有轮询远端 checks。完整 54-mutant suite 本轮未重跑;对这个单一 Case 映射而言,targeted control+mutation 和 workflow/premerge 覆盖已经足以证明改动。PR 当前落后于 main,若 rebase 改变 head,必须重新审阅,不能继承本结论。
同作者近期开放的 #4859、#4860、#4873 分别是 fixture、full-smoke 与研究文档边界,不是同形态 smoke farming;本 PR 没有新增 walkthrough 或重复 fixture,而是把现有 mutation 绑定到现有更直接的 oracle,具有持续的仓库质量价值。
我的整体评价
APPROVE。设计上选择了更小、更接近不变量所有者、且能被 deliberate mutant 真实击穿的边界;实现只调整 oracle 映射,没有形成第二套权威或隐藏生产行为变化。未来若该 TypeScript 测试被重命名或退休,mutation runner 的 locator/control gate 会显式失败,维护责任清晰。
English verdict: APPROVE - head 6dcb8c763021750011d9b96c329f85e8571fb126; the existing in-process TypeScript state-lock oracle passes unchanged and kills the exact unshared-lock mutant by assertion. Targeted mutation/control, 44 workflow tests, Ruff, diff hygiene, and the 6-check premerge gate passed; remote CI was not consulted by Goal policy.
Summary
fence_unshared_state_lockto the existing in-process TypeScript state-lock ordering testEvidence
The #4859 CI artifact executed all 54 mutants but killed only 53;
fence_unshared_state_locksurvived because the cross-process oracle passed unchanged. The same full suite kills 54/54 locally with this mapping.Verification
tests/test_python_ci_workflow.py: 44 passed