Skip to content

🤖 feat(tasks): fence sub-agent attempts before durable workflow recovery - #4308

Open
ThomasK33 wants to merge 15 commits into
mainfrom
thomask33/workflow-attempt-admission
Open

ThomasK33 wants to merge 15 commits into
mainfrom
thomask33/workflow-attempt-admission

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Sep 20, 2026

Copy link
Copy Markdown
Member

Summary

Establish the attempt identity and send-admission boundary needed for workflow recovery across backend restarts. This is G1, the prerequisite layer. Durable settlement producers, cross-restart classification, retirement claims, and child replacement belong in the dependent G2 PR.

All implementation and tests were generated with Xum. This disclosure includes earlier source commits without individual attribution footers.

Review focus

  1. Publishing admissions receive immutable attempt IDs; unproven lineage stays unproven. Config preserves identity, lineage, and the future retirement-claim field.
  2. Settlement closes admission before awaited work. Admission tokens follow preparation, queuing, actual turn admission, and cancellation. Stop waits for captured turns, executions, and pending admissions.
  3. Manual send/Resume after stopping a reported child's continuation starts a new attempt, preserving historical report metadata. The old attempt stays closed. Failed reactivation never rolls back its published identity.
  4. Receipt parsing/storage infrastructure is present, but no TaskService receipt producer is enabled.

Validation and status

Current head: 0333deb1c864d10845e191e74c0a490de932071e, a normal merge of pinned main (60d40394d) into the earlier UAT snapshot. The constants conflict is resolved without rewriting history. Exact-head integration checks passed: G1/TaskService 908, WorkspaceService 652, AgentSession 1012, workflow/tool/config 248, IPC 9, and make static-check. All 20 CI check-runs are successful or skipped; optional Pixel visual approval remains pending. Exact-head remote integration UAT passed. Codex review is now being requested. No merge is requested.

The integration smoke independently verified closed-reported chat-box recovery and Resume, retained report metadata, new attempt IDs, and no dispatch of stale queued guidance. It also covered the newly integrated unrelated-recipient consent gate: default-off refusal, real-UI opt-in, Stop/revoke-before-dequeue non-dispatch, and fresh delivery after re-enabling. Evidence includes a recount of all 77 raw fixture request bodies. The first unrelated-Stop run completed before Stop and is excluded; a later correctly timed run supplies that evidence. One fresh-after-recovery send was only observed through admission because its default provider lacked a key; the subsequent fresh-send scenario proves provider delivery. No paid provider calls were made.

Exact-head integration evidence at 0333deb

Chat-box recovery on the integrated head

Integrated recovery at 390px

Independent Resume from a closed reported attempt

integration-scenario1-stop.webm

Full receipts: .mux-uat/integration-60d40394d/remote-uat/status.json, request-body recount, screenshots/video, and artifact hashes. The remote backend was restarted after verifying rebuilt-module timestamps. Remote servers and workspace were stopped after capture.

  • At the pre-integration UAT snapshot, Bun 1.3.5: G1 158 tests; TaskService 656; WorkspaceService 640; AgentSession 958; workflow/tool/config siblings 247. Exact-head make static-check passed.
  • Real IPC: both reported-child recovery paths reproduced red; all nine tests passed after the fix. The clean-clone wrapper initially rejected its dependency symlink after Jest exited 0; that separate cleanliness guard was corrected. One initial TaskService run had a temporary-lock cleanup error; an unchanged rerun passed. First-failure logs remain preserved.
  • Remote UAT passed actual chat-box send and independent Resume from closed reported attempts, three repeated cycles, queued-input cancellation, interrupted recovery, and seeded retirement refusal. New IDs and retained report metadata were checked. Desktop/390px screenshots and video were inspected.
  • Remote chat; local receipts: .mux-uat/round-6/. Media below belongs to daed4432d, not a future integration head.
Recovery evidence at daed443

Manual follow-up accepted after parent cascade Stop

Accepted follow-up at 390px, drawer closed

r6-recovery.mp4

Video covers independent Resume and a repeated chat-box cycle; the primary chat-box run has desktop/phone PNGs. One mid-stream screenshot belongs to the separate Resume child; fixture request/abort logs prove the primary Stop. Six harness scripts listed in the remote manifest were not attached; their captured outputs are retained. These evidence limitations are recorded separately from the passing recovery checks.

Risks and limits

The main risks are refusing legitimate sends or retaining Stop obligations indefinitely. Regression tests cover those branches and concurrent Stop, identity, status, settlement, and claim changes. Legacy/unproven attempts still lack cross-process proof. Reactivation-owned turns still preserve stable reported status and are not startup-re-driven after backend death; that pre-existing lifecycle ownership issue is not fixed here.

Follow-up: this workspace owns G2 after G1 approval, as the upper native stacked PR. The broader reactivation/startup lifecycle correction remains a separately scoped follow-up, triggered when restart recovery for reactivated reported children is undertaken.

Implementation plan

The complete approved two-layer plan is 100,478 characters and exceeds GitHub's PR-body limit. Part 1 and part 2 preserves the complete text verbatim in numbered comments. Below are verbatim G1 excerpts; the full plan's G2 work is not part of this diff.


G1 implementation plan — verbatim excerpt

Change 1 — Immutable attempt identity

  • WorkspaceConfigEntry.taskAttemptId?: string and taskAttemptRetiredBy?: { runId: string; stepId: string; inputHash: string; childTaskId: string; attemptId: string; mode: "no-report" | "retire-reported"; at: string } (src/node/config/index.ts type near :197, parse site near :1104, the three explicit field lists). Both optional; absent on legacy entries.
  • Third optional field taskAttemptUnproven?: true (same three field lists): present when the attempt's lineage is not proven (P11). Written together with taskAttemptId by the admission that rotates the id; omitted by the reservation commit and by proven reawaken/reactivation; inherited (copied) by every admission that does not itself prove the lineage. Never cleared except by a new reservation (a new task).
  • OwnedTaskAttempt (taskService.ts:443–452) gains readonly attemptId: string | undefined and readonly receiptEligible: boolean; beginOwnedTaskAttempt(taskId, source, { abortSignal?, attemptId, receiptEligible }). receiptEligible is decided once, at admission, from the evidence in P11; it is never recomputed later.
  • Proof rule at reawaken/reactivation (one helper evaluateAttemptLineage(taskId, entry): Promise<{ proven: boolean; reason: string }>, evaluated under the task's event lock before the admission CAS): proven iff entry.taskAttemptId != null && entry.taskAttemptUnproven !== true and either (i) ownedAttemptByTaskId.get(taskId) has that attemptId, receiptEligible === true, and attemptSettlementByTaskId records its settlement — if the owned attempt is still settling (stop latch retained), wait with the existing bounded waitForAttemptSettlement first, then re-evaluate; or (ii) readSubagentAttemptSettlementReceiptStrict(parentDir, taskId, entry.taskAttemptId) is found (unreadable/not_found → not proven). The bounded waitForAttemptSettlement runs outside the task event lock (it waits for publication/settlement work that needs the lock); the lock is then reacquired and the proof re-evaluated. The admission CAS publishes taskAttemptId = newId and decides the marker from the fresh transaction row: proven = precomputedProven && ws.taskAttemptUnproven !== true (the marker is only ever added on a given task, so a marker that appeared after the snapshot can only downgrade). The mutator records its committed result, and beginOwnedTaskAttempt(…, { receiptEligible }) is called with that committed value — never with a pre-await snapshot. Unproven admissions proceed exactly as on main (owned in memory; no receipts) and log the reason once.
  • Id generation: att_ + 16 hex chars from crypto.randomBytes (src/node/utils/… helper next to existing id helpers); assert(/^att_[0-9a-f]{16}$/.test(id)) at every write and read.
  • Writers (each inside the existing config mutation that performs the admission, so id and status change atomically):
    • reservation commit in createMany (the editConfig that persists reserved plans, near :4296): taskAttemptId = newId, no taskAttemptUnproven; the OwnedTaskAttempt created at :4296 carries it with receiptEligible: true (the reservation is the first admission by construction).
    • startReservedAgentTask (:4758–5072): keeps the reservation's id and eligibility; if the entry has none (pre-upgrade queued/starting entry), assign one in its existing status write with taskAttemptUnproven: true (an old build's stale-starting revert may have reverted an admitted entry to queued without any marker) and carry it into beginOwnedTaskAttempt("launch", { receiptEligible: false }).
    • markInterruptedTaskRunning (:12237–12296): strict pre-read refuses when taskAttemptRetiredBy != null (returns false, before beginOwnedTaskAttempt); evaluateAttemptLineage decides proven; the mutator re-checks (if (ws.taskAttemptRetiredBy != null || ws.taskAttemptId !== previous) return;) and sets taskAttemptId = newId, taskAttemptUnproven per the proof, with running. On mutator refusal, restore the previous in-memory ownership/settlement exactly as reactivateChildAgentTask does (:5911–5925).
    • reactivateChildAgentTask (:5886–5930): before createWorkspaceTurn, evaluateAttemptLineage, then an editWorkspaceEntry CAS (taskAttemptId === previous && taskAttemptRetiredBy == null) publishes taskAttemptId = newId and taskAttemptUnproven per the proof (same lifecycle lock), then beginOwnedTaskAttempt("reactivation", { attemptId, receiptEligible: proven }). Boundary: if the CAS fails (claimed or id changed), nothing was published — restore speculative memory as today (:5911–5925) and return the refusal. Once the CAS committed, the new identity is kept in both config and memory whatever createWorkspaceTurn returns or throws: the old attempt is never restored after its successor is published. A refused/failed reactivation therefore leaves an owned, unsettled attempt (indeterminate, reason "owned attempt without settlement evidence"); a subsequent Stop settles it through the idle or execution-settlement producer and writes its receipt. The existing in-memory restore block is reduced to the pre-commit case.
    • Launch ownership for plans this process did not reserve (queue drain maybeStartQueuedTasksFromReservations, :12082–12084, which today sets starting unconditionally): the transition becomes a CAS taskStatus === "queued" && taskAttemptRetiredBy == null → starting that also rotates taskAttemptId when ownedAttemptByTaskId.get(id)?.attemptId !== entry.taskAttemptId and copies taskAttemptUnproven unchanged; the process whose CAS commits is the single owner (beginOwnedTaskAttempt("launch", { attemptId, receiptEligible }) with receiptEligible = ws.taskAttemptUnproven !== true read from the fresh row inside the mutator, not from the drain's snapshot — a stale-starting revert by another process can add the marker without changing the id), the loser skips the plan. A queued entry without the marker has by definition never been admitted (admission persists starting first, and the only path back to queued sets the marker below), so this handoff is exclusive and proven by construction.
    • Stale-starting revert (:3198–3211): the mutator additionally sets taskAttemptUnproven = true when it writes queued (or running). A starting entry found at startup may already have an admitted execution in another process; the marker makes the relaunch — which otherwise looks exactly like a never-launched reservation — an unproven lineage, so the drain's launcher owns it in memory as today but is never receipt-eligible. Entries a process finds in running or awaiting_report may likewise have an admitted execution elsewhere: the existing re-drive keeps running unowned — no beginOwnedTaskAttempt, no receipts — and marks the lineage unproven (below), so no receipt can ever describe a child that another process may still publish for, and nothing reawakened from that lineage can produce one either.
    • Startup re-drive (recoverInterruptedTasks: guidance replay :3374, restart nudge :3417, promptTaskForRequiredCompletionTool({ reason: "startup" }) :3317, dispatchPendingCompactionFollowUp :3313/:3358): before the send, an editWorkspaceEntry CAS rotates taskAttemptId and sets taskAttemptUnproven = true (refusing when taskAttemptRetiredBy != null, in which case the task is skipped and logged). The task stays unowned (no beginOwnedTaskAttempt, no receipts), any receipt of the previous attempt can no longer match the execution this process starts, and every later reawaken of this child inherits the unproven marker.
    • Within-owner continuations of the same attempt do not rotate: in-owner recovery prompts (promptTaskForRequiredCompletionTool :12466/:12539 from :13127, :13366, :13707, :14390), parent guidance queued into a live child (:6192, :6993), best-of/continuation kickoffs of an owned attempt (:9460:10020, :12823:12876, :13848). Rule, enforced by one helper admitTaskSend(taskId, { newAttempt }) used at every send site: if the attempt is admission-openownedAttemptByTaskId.get(taskId)?.attemptId === entry.taskAttemptId, and attemptSettlementByTaskId has no entry for it (neither closing nor settled, Change 3), and no workspaceStopRecords entry exists for the workspace, and taskAttemptRetiredBy == null — the send continues the same attempt (no rotation) and is registered as an AdmittedSend obligation bound to that attempt until discharged (Change 3); if the attempt is owned but settled or settling, the send is refused (Err("attempt settled"), logged; the caller drops it — an intentional continuation after settlement is a reawaken and must go through the fresh-id CAS); otherwise the helper performs the rotation CAS (newAttempt: { own: true, receiptEligible } also begins ownership; newAttempt: { own: false } is the unowned startup re-drive, which rotates and marks but never owns), refusing when claimed. Settlement therefore closes an attempt to further sends, so no continuation can dispatch under an id that a receipt already describes (P1/P4). Uses existing settlement/stop state only; no new persisted field. Enumerate every workspaceService.sendMessage / prompt call on a task workspace in taskService.ts (the line list above is the seed), route each through the helper, record the classification in a comment at each site (test 1b covers each).
    • Dispatch-time identity check. A committed CAS does not cancel a send that was already prepared. Every task send carries expectedAttemptId; the admission fence Layer 2 added before the provider start (the check "at physical turn admission, immediately before sendMessage" used by startReservedAgentTask, and the AgentSession/StreamManager provider-start fence) re-reads the entry under the task's event lock and aborts the send when taskAttemptId !== expectedAttemptId or taskAttemptRetiredBy != null; the authoritative in-flight check is the existing admissionStale probe (SendMessageInternalOptions, evaluated synchronously before coordinator.prepare and at dequeue) supplied by admitTaskSend as () => !attemptAdmissionOpen(taskId, expectedAttemptId) — a continuation prepared before a Stop is refused at the session gate, never dispatched after the receipt (Change 3). Implementation verifies the exact seam (the fence added for TaskLaunchPlan.abortSignal) and reuses it rather than adding a second fence.
  • No other code reads taskAttemptId for behavior; it is identity only.

Change 2 — Receipt module (src/node/services/subagentAttemptSettlements.ts, new)

  • Path: sessions/<ownerWorkspaceId>/subagent-attempt-settlements/<encodeURIComponent(taskId)>/<attemptId>.json, written into the parent and every ancestor session dir (same fan-out as failure artifacts; ancestors resolved from the captured entry, not re-read).
  • Record: { version: 1, taskId, attemptId, parentWorkspaceId, source, settledAt } with source ∈ { "execution-settled", "idle-settled", "launch-failed", "reservation-canceled", "reservation-failed" } (terminal failures appear as execution-settled or idle-settled depending on the producer that wrote them; the failure artifact keeps the error detail).
  • writeSubagentAttemptSettlementReceipt(params): Promise<Result<void, string>> — temp file + rename; if the target exists, verify equal attemptId and return Ok (idempotent). Never throws; it settles only when the OS write settles (no internal timeout), so ownership of the write is never abandoned.
  • readSubagentAttemptSettlementReceiptStrict(ownerDir, taskId, attemptId): Promise<{ kind: "found"; receipt } | { kind: "not_found" } | { kind: "unreadable"; error }> — ENOENT → not_found; any other error or schema failure → unreadable.
  • Deletion: none (immutable; bounded by attempts that ended without a report, like the in-memory receipts).

Change 3 — Admission lifecycle and receipt writes (taskService.ts, workspaceService.ts, agentSession.ts, messageQueue.ts)

private async persistOwnedAttemptSettlement(taskId, attempt: OwnedTaskAttempt | undefined, source): Promise<boolean> — returns false without writing when attempt == null, attempt.attemptId == null, attempt.receiptEligible === false (unproven lineage, P11), or this.ownedAttemptByTaskId.get(taskId) !== attempt (the same guard settleOwnedTaskAttempt applies, evaluated before the write); otherwise writes the receipt for attempt.attemptId and returns whether it became durable. Then the caller calls settleOwnedTaskAttempt as today (in-memory settlement is recorded even if the write failed; the log line distinguishes durable: true|false). It is called only from the producers in 3b, which are enabled only after gate G1 (3a).

Change 3a — Admission lifecycle contract (prerequisite; gate G1 must be green before any receipt producer is enabled)

Linearization point — admission closes before the first awaited receipt/config operation (P1/P4). Today no producer marks an attempt as settling before its awaited write (OwnedTaskAttempt is { generation, source, abortSignal? }, immutable; attemptSettlementByTaskId is populated only after editWorkspaceEntry/cleanup awaits in all six callers), and the launch fence at startReservedAgentTask:5028–5045 holds no lock, so a send prepared before a Stop could dispatch during the receipt write. The fix reuses the existing settlement map and stop records and adds one in-memory obligation set plus one explicitly planned integration: a TurnAdmissionToken carried in the send options through WorkspaceService, AgentSession and the MessageQueue (the WorkspaceService-facing SendMessageInternalOptions does not expose onTurnAdmissionCommitted today — that callback is AgentSession-internal and releases WorkspaceService's preflight reservation; the token is forwarded alongside it and composes with it, never replaces it). Changes 1–2 and 3a ship and are verified together (G1, receipt-independent — see Quality gates) before 3b–5 enable receipts and claims. A partially wired token fails closed during development (an obligation that can never be dispositioned keeps the stop latch retained → cleanup-pending, visible, never a receipt) — that is a failing G1, not a passing one.

  • attemptSettlementByTaskId entries become { attemptId: string | undefined; attempt?: OwnedTaskAttempt; phase: "closing" | "settled"; source }. closeAttemptAdmission(taskId, { attemptId, attempt }, source) records the closing entry synchronously, under workspaceEventLocks.withLock(taskId), before the producer's first await, with attemptId captured from the fresh config row inside the producer's updater (so unowned attempts are closed for their own id too; attempt only when owned); settleOwnedTaskAttempt upgrades the same entry to settled (its existing owned-guard unchanged). beginOwnedTaskAttempt/a new-attempt admission deletes the entry (a fresh id reopens admission). An entry in any phase is never removed by a failed write: admission stays closed; intentional continuation is a reawaken (fresh id). Every consumer that treated settlement-map membership as settlement now requires phase === "settled": the owned path of inspectAttemptOutcome (:2369–2377, closingcleanup-pending), waitForAttemptSettlement, evaluateAttemptLineage (i), the claim's same-process evidence, and handleAgentWaitFailure's consumers through the adapter.
  • Send obligations with immutable identity. admittedSendsByTaskId: Map<taskId, Set<AdmittedSend>>, AdmittedSend = { readonly attemptId: string; readonly attempt?: OwnedTaskAttempt; state: "pending" | "enqueued" | "admitted" | "discharged"; turnId?: TurnId }attemptId is the captured config id the send was admitted under (non-optional, also for unowned startup sends); attempt only for owned admissions (receipt authority). The obligation is created by the fence before dispatch and dispositioned only through its TurnAdmissionToken; the send call's return value is never used as evidence. Discharge updates the obligation object and the record sets that captured it by reference, never whichever record currently occupies the task's map slot; an obligation admitted under attempt X can never enter, or be removed from, a successor attempt's record.
  • TurnAdmissionToken (new field on the WorkspaceService-facing SendMessageInternalOptions, taskWorkspaceSeam.ts:330–370, forwarded into AgentSession's internal options agentSession.ts:745–831 and stored on the MessageQueue entry exactly like the existing admissionStale): { admissionStale(): boolean; onAdmitted(turnId: TurnId): void; onDisposed(kind: "no-work" | "refused" | "canceled-before-admission"): void }. onAdmitted is idempotent per turnId (a dequeued item is prepared at :10128 and then adopted by sendMessage(…, { turnReservation }) at :10188 — the second call must not create a second admission); onDisposed is idempotent and ignored once admitted. Fired at the first actual preparation admission of every path, never inferred later:
Seam (verified) Token event Obligation
workspaceService.sendMessage pre-queue/pre-session Err sites (:11696–11844, :11864, :11939/:11971, :12005–12010, :12051–12063, :12188–12190) and agentSession.sendMessage pre-prepare refusals (:3338, :3669/:4293, :4300, the admissionStale gate :4791–4805, prepare rejected/deferred :4834–4842) onDisposed("refused") discharged (never admitted)
dedupe / restore returns (:11759, :11763) and resumeStream Ok({ started: false }) (busy :4970, closing :5004, prepare rejected :5039) onDisposed("no-work") discharged — no work exists for this token; the deduped-into item keeps its own token
session.queueMessage (:12127–12159) (none yet; entry stores the token) enqueued — stays until its own dequeue
dequeue sendQueuedMessages (:10097–10201): before coordinator.prepare at :10128 evaluate the item's admissionStale (added at that gate if absent) → stale ⇒ item dropped, onDisposed("refused"); else prepare admitted ⇒ onAdmitted(turnId) here, before the sendMessage(…, { turnReservation }) re-entry at :10188 as stated admitted from :10128 on — an early failure inside the :10188 re-entry is not "never admitted"; its turn settles through the preparation-failure path
direct coordinator.prepare admitted (agentSession.ts:4817; existing onTurnAdmissionCommitted :4846 keeps releasing WorkspaceService's preflight reservation) and resumeStream prepare admitted (:5030) onAdmitted(turnId) with the admitted id passed explicitly admitted
streamWithHistory failures after admission (:7480, :7483, :7736, :7813/:7879, resume :5071) (none) stays admitted; discharged only when that turn settles (settlePreparationFailure → turn completion → recordWorkspaceTurnSettled; implementation verifies the turn-settled event fires for failed-start turns, otherwise extends that seam)
cancellation (onCanceled, cancelSignal, clearQueue via Stop taskService.ts:7227/:7726 → workspaceService.ts:2078, interruptStream :12879 → agentSession.ts:10028) onDisposed("canceled-before-admission") only if not yet admitted pending/enqueued → discharged; admitted (PREPARING or later) → retained until the correlated turn settles — cancellation requests termination, it does not prove settlement
turn settlement recordWorkspaceTurnSettled(turnId) (TaskService-internal) admitted with that turnId → discharged; removed from every capturing record's capturedTurns
supersession (a new taskAttemptId published) (none) existing obligations keep their attemptId; admitted ones are retained until settlement; enqueued ones are refused at dequeue by their own admissionStale

token.admissionStale = () => token.state === "discharged" || !attemptAdmissionOpen(taskId, token.attemptId) || (token.attempt != null && ownedAttemptByTaskId.get(taskId) !== token.attempt) reads TaskService in-memory state only and is the refusal authority: evaluated synchronously at the enqueue block (:12051, :12188), at the session gate (:4791) and at dequeue (:10128), it returns Err before any coordinator state changes. The common predicate attemptAdmissionOpen(taskId, id) (send authorization, no ownership) requires: currentAttemptIdByTaskId.get(taskId) === id — an in-memory mirror of the task's current taskAttemptId, updated synchronously by every local rotation (reservation commit, drain launch CAS, reawaken, reactivation, startup re-drive rotation, owned or not), so any later local rotation revokes every older pending/enqueued token by construction, including unowned startup generations (startup Y in preflight → startup Z rotates → Y reaches prepare is refused); no closure entry for id; no workspaceStopRecords entry. Ownership is checked separately and only where it applies: a same-attempt continuation requires ownedAttemptByTaskId.get(taskId)?.attemptId === id at the fence and, through token.attempt, at every later gate; a new owned attempt passes the common predicate for the just-published Y first and then, in the same synchronous block, installs ownership (beginOwnedTaskAttempt(Y)) and registers its obligation with token.attempt set — so owned X settled → CAS publishes Y → admission of Y succeeds (the previous owned X never blocks Y); a startup re-drive passes the common predicate and registers an unowned obligation (token.attempt undefined), never acquiring ownership to satisfy any check. beginOwnedTaskAttempt(newId) deletes the settlement entry only when entry.attemptId !== newId: a closure recorded for Y between the CAS and the block makes the block refuse and is never erased. A disposed token is terminal: onDisposed sets state = "discharged" and the token can never admit afterwards even if its attempt stays open. admitted tokens are unaffected by rotation (they are retained until their turn settles). So Y passes the fence → awaits → Stop closes Y → prepare and Y → Z supersedes → Y prepares are both refused. There is no "send settled + workspace idle" discharge and no inference from Ok/Err.

  • Stop record. WorkspaceStopRecord gains attemptId (the captured attempt's id; for an unowned attempt the task index's current id at capture), pendingAdmissions: Set<AdmittedSend> (the exact obligation objects for that attemptId, captured by reference in Phase A) and capturedTurns: Set<turnId> (replacing the single capturedTurn; initialized from the current turn). Release (recheckWorkspaceStopRelease) requires stopPersisted && capturedTurns.size === 0 && pendingAdmissions.size === 0 && executionSettled && cleanupInFlight === 0. When a pending admission's turn becomes visible, the record adds it to capturedTurns and owes one more cleanup (cleanupInFlight += 1) for a second aiService.stopStream(taskId) so the late turn is actually stopped; its settlement removes it. A captured turn settling while another admission is pending therefore cannot release the record (the round-10 counterexample: T1 settles, receipt, then admitted T2 starts).
  • Per producer: idle stop (releaseSharedDesktopTaskOnUserStop, already under the event lock :2692–2695; idleness verified inside the updater :13221–13227) and failAgentTaskTerminally (under the event lock in every streaming caller :2704–2707, :10077; the startup caller :3317 acts on an unowned task and writes no receipt) treat any pending admission as live (→ stop-record route, never an idle receipt) and otherwise call closeAttemptAdmission right after their idleness decision, before editWorkspaceEntry. The stop cascade's Phase A record creation (beginWorkspaceStop :2017–2050, synchronous under the global mutex, ownedAttempt captured) is the closure for the execution-settlement producer: the admission-open predicate treats an existing workspaceStopRecords entry as closed. Reservation canceled/failed: the plan's abortSignal is aborted before the write and the launch fence's abortSignal.aborted check (:5028–5033, the existing linearization comment) refuses the launch — no new marker. markTaskLaunchFailed: takes the event lock, calls closeAttemptAdmission, and writes a receipt only if plan.sendAdmitted === false and no admission is pending for the attempt, decided inside that lock.
  • Receipt I/O runs outside the event lock on every path. Idle/terminal producers, under the lock: idleness decision → closeAttemptAdmission → the existing editWorkspaceEntry → register the receipt write as an owed cleanup bound to the captured attempt → release the lock. The write then runs outside; on completion (fulfilled or rejected) settleOwnedTaskAttempt(taskId, attempt, source) upgrades the entry to settled against the captured identity (durable: true|false). While the write is pending, inspectAttemptOutcome (which takes the same lock) sees closing and returns cleanup-pending, and a fence call returns a refusal immediately rather than queueing on the lock. The stop path keeps P7's owed-cleanup form.
  • Fence (admitTaskSend, Change 1) runs under workspaceEventLocks.withLock(taskId) up to dispatch. Its awaited step is the strict config read (continue same attempt) or the rotation CAS that publishes the fresh id (new attempt: reawaken, reactivation, drain launch, startup re-drive). Then, synchronously: expectedId (X for a continuation, the just-published Y) equals the row's taskAttemptId; taskAttemptRetiredBy == null; the common attemptAdmissionOpen(taskId, expectedId) (generation current, no closure entry for expectedId, no workspaceStopRecords entry); for a continuation only, ownedAttemptByTaskId.get(taskId)?.attemptId === expectedId. Only then, still in the same synchronous block: owned new attempts beginOwnedTaskAttempt(expectedId, { receiptEligible }); register AdmittedSend { attemptId: expectedId, attempt?, state: "pending" } (attempt set for owned admissions, undefined for startup re-drives); set plan.sendAdmitted = true; call sendMessage/resumeStream with the obligation's TurnAdmissionToken. The pre-dispatch checks are an early exit; the authoritative refusal is the token's admissionStale, evaluated by the session synchronously before every coordinator.prepare (direct :4791, dequeue :10128, resume) so the asynchronous work inside sendMessage (compaction admission, pricing gate, preflight, history publish — workspaceService.ts:11860–11982, agentSession.ts:4321–4370) cannot admit a stale identity: Y passes the fence → awaits → Stop closes Y → prepare is refused at :4791. A refusal after a committed CAS leaves the fresh id published (P3: no rollback), the owned-but-never-admitted attempt readable as indeterminate (owned, unsettled) until a Stop settles it with its receipt, and returns Err. Send authorization is separate from receipt authority: startup re-drives pass the new-attempt branch unowned (fresh marked id, no beginOwnedTaskAttempt, no receipts), exactly the behavior Change 1 preserves, but they cannot send through a concurrent Stop or a closed attempt; owned branches acquire receiptEligible only per P11. closeAttemptAdmission records the closed attemptId also for unowned attempts (captured from the fresh row inside the producer's updater), so an unowned startup attempt that is Stopped is closed for its own id as well.
  • Queued input. Dequeue does not re-enter workspaceService.sendMessage (sendQueuedMessages, agentSession.ts:10097–10201, calls coordinator.prepare at :10128 and agentSession.sendMessage at :10188 directly), so the queued item's own token is the fence there (table above), together with Stop semantics: the task stop cascade runs runWorkspaceStopCleanup with clearQueue: true (taskService.ts:7227, :7726workspaceService.clearQueue, :2078), sendQueuedMessages freezes while a stop latch is held (agentSession.ts:10109), and interruptStream restores the queue to input (workspaceService.ts:12879clearQueue, agentSession.ts:10028). Cleared, not-yet-admitted items are dispositioned canceled-before-admission. The idle/terminal producers add workspaceService.clearQueue(taskId) to their closure step (the terminal-failure stop-record route already gets it from Phase B), so no deferred dispatch can outlive a closure; a bare aiService.stopStream is never used as a settlement producer (it does not touch the queue).

Generated with xum • Model: coder:openai/gpt-6-astra • Thinking: xhigh • Cost: $588.43

…t module

Persist taskAttemptId/taskAttemptUnproven/taskAttemptRetiredBy on task config
entries (identity only; no consumer branches on them yet), preserve them across
addWorkspace metadata round trips, add the att_ id helper and the immutable
per-attempt receipt module (temp+rename, strict reads). No receipt producer or
classifier is wired in this layer.

Signed-off-by: Thomas Kosiewski <tk@coder.com>
Every admission that can start a publishing execution now rotates a persisted
taskAttemptId in the same config write (reservation commit, exclusive queued→
starting launch CAS, reawaken, reactivation, unowned startup re-drive) and
records whether the lineage is proven (taskAttemptUnproven / receiptEligible).
Sends into task workspaces carry a TurnAdmissionToken minted by TaskService at
the WorkspaceService handoff, admitted inside the coordinator's synchronous
prepare callback, refused at the dequeue gate before any turn is claimed, and
discharged only when their turn settles or is superseded. Settlement producers
close the attempt synchronously before their first awaited write; stop records
wait on pending admissions and every captured turn (rebound on supersession).
No receipt producer, classifier or claim is enabled.

Signed-off-by: Thomas Kosiewski <tk@coder.com>
…s for the G1 layer

Adds real-WorkspaceService tests for the task-attempt fence at the session
handoff (refusal message, caller-minted token reuse, queue handoff, dedupe
before the fence, resume admission), records the admission classification at
every TaskService send site, disposes a refused resume as refused rather than
no-work, and skips the unowned stop closure for pre-identity entries.

Signed-off-by: Thomas Kosiewski <tk@coder.com>
While a producer's closing config write is in flight the owned attempt reads
cleanup-pending and the fence refuses continuations; the entry upgrades to
settled once the write completes.

Signed-off-by: Thomas Kosiewski <tk@coder.com>
…n the launch-failure test

The in-memory settlement follows the persisted status asynchronously; the
closing window between them is covered by its own test.

Signed-off-by: Thomas Kosiewski <tk@coder.com>
… sub-agent suite

A reactivation publishes its fresh attempt before createWorkspaceTurn, and a
refusal there no longer rolls the identity back to the retired attempt: the
task reads owned-but-unsettled (indeterminate) until a Stop settles it as
terminal-no-report with the published id unchanged. The taskService unit tests
already encode this; the ipc suite still asserted main's immediate
terminal-no-report and failed under TEST_INTEGRATION.

Signed-off-by: Thomas Kosiewski <tk@coder.com>
…Stop cascade

Remote G1 UAT (round 3, criterion 6): a parent hard Stop landing on a child
reawakened via task_send_message left the child's stop latch held until
restart; every later send was refused with "A stop is in progress for this
workspace; retry once it has settled."

Root cause: terminateAllDescendantAgentTasks captures the child's live
WorkspaceTurnManager registration (capturedExecutionId) and waits for its
settlement, but Phase B stops the stream with a "system" abort, which never
settles a continuation handle (finalizeWorkspaceTurnFromStreamAbort settles
only user aborts). The mirror stayed "running", releaseRetainedStopLatches
never ran, and the record read cleanup-pending forever.

Fix: Phase A also captures the live registration's owner + handle; each
target's bounded Phase B cleanup now interrupts that handle
(WorkspaceTurnManager.interruptWorkspaceTurn) and suppresses the owner's
terminal wake before the stream stop - the same pairing task_stop's subtree
stop already uses - so the captured execution settles authoritatively and the
latch drops once the streaming generation settles. Deadlines and fail-closed
retention are unchanged.

Tests: reactivation -> cascade -> mirror interrupted -> latch releases ->
reawaken admitted again (G1 suite); cascade over a reported reawakened child
settles handle, mirror, registration and attention (taskService suite). The
three retained-latch tests now model the fail-closed case where the explicit
interrupt fails.

---

_Generated with `xum` • Model: `coder:anthropic/claude-fable-5-1` • Thinking: `xhigh` • Cost: `$23.45`_

<!-- mux-attribution: model=coder:anthropic/claude-fable-5-1 thinking=xhigh costs=23.45 -->
Behavior-neutral cleanup of the G1 diff (32834cc..6e85cc2); no gate,
assertion, proof comment or test assertion changes.

- closeAttemptAdmission takes (attemptId, ownedAttempt) and applies the
  owner-match predicate itself; the three settlement producers (launch
  failure, idle user stop, terminal failure) no longer repeat the same
  8-line identity object, and the stop-settled call passes no owner.
- MessageQueue.removeEntry drops its disposition parameter: the dequeue
  gate is its only caller and always disposes the token as refused.
- Reawaken lost-CAS comment corrected: reactivation begins its attempt only
  after its CAS commits, so it has no speculative ownership to undo.

Validation (Bun 1.3.5): taskService.attemptAdmission 19, settlements 5,
agentSession.turnAdmission 7, workspaceService.turnAdmission 6,
messageQueue 109; taskService 656, workspaceService 640, agentSession 958,
tools/task 149; make static-check exit 0.

---

_Generated with `xum` • Model: `coder:anthropic/claude-fable-5-1` • Thinking: `xhigh`_

<!-- mux-attribution: model=coder:anthropic/claude-fable-5-1 thinking=xhigh -->
…ver a reactivated child

Remote G1 UAT (round 5): after a parent reawakened a completed child via
task_send_message and a parent Stop cascade settled that reactivation attempt,
both manual recovery paths (chat send and workspace.resumeStream) were refused
forever with "This sub-agent's current attempt has settled; resume it
explicitly to start a new attempt."

Encode the contract on the real path (TaskService + WorkspaceService +
WorkspaceTurnManager, mock AI, IPC entry points) for both predecessor statuses:

- interrupted predecessor: passes today (markInterruptedTaskRunning mints a
  fresh attempt) and is now pinned.
- reported predecessor: reproduces the refusal on both entry points. The
  reactivation never publishes an active stable status, the cascade's
  applyInterruptedTaskStatus preserves `reported`, markInterruptedTaskRunning
  refuses to mint for a non-desktop reported child, and the admission fence
  keeps refusing the settled reactivation attempt. Pinned with
  test.failing.each until the lifecycle correction lands; a fix flips these
  cases to plain test.each. No production change in this commit.

---

_Generated with `xum` • Model: `coder:anthropic/claude-fable-5-1` • Thinking: `xhigh` • Cost: `$15.58`_

<!-- mux-attribution: model=coder:anthropic/claude-fable-5-1 thinking=xhigh costs=15.58 -->
A parent continuation leaves the persistent child reported. After cascade
Stop settled that continuation, manual send and Resume could not mint a new
attempt, so the closed-attempt fence rejected both forever.

Allow explicit manual recovery only with same-process settled evidence for
the exact current attempt. Rotate identity while keeping reported status and
reportedAt. Recheck Stop, identity, status, claim, and settlement evidence
across the awaited boundary; never reopen the old attempt.

Validate both real IPC recovery paths, old-token refusal, narrow eligibility,
concurrent changes, and existing interrupted/desktop behavior. Receipt
producers and the broader WTM lifecycle remain outside this G1 repair.

---

_Generated with [`xum`](https://github.com/coder/xum) • Model: `coder:openai/gpt-6-astra` • Thinking: `xhigh` • Cost: `$533.73`_

<!-- mux-attribution: model=coder:openai/gpt-6-astra thinking=xhigh costs=533.73 -->
@ThomasK33

Copy link
Copy Markdown
Member Author

Approved implementation plan (1/2)

Full-plan SHA-256: b3c920d8666fc96eebcc9bbc64d21a6192bc8fd2f1c0e5970a0ff0be9825c029. Records G1 and future G2; review only G1 here.

Verbatim plan, part 1

Workflow recovery across restarts: owner-bound durable settlement receipts with claimed retirement

Result first

After a backend restart, a workflow started step that points at a child from the previous process can never be classified terminal-no-report: TaskService.inspectAttemptOutcome proves settlement only from two process-local ledgers. WorkflowRunner.classifyPriorAttempt then throws WorkflowPriorAttemptUnresolvedError, the run is re-interrupted, and nothing replaces the dead child.

This change is an explicitly partial improvement, not complete cross-restart recovery:

  1. Immutable attempt identity with proven lineage. Every admission that can start a publishing execution (reservation, reawaken, reactivation, first launch of a pre-upgrade entry) persists a fresh opaque taskAttemptId in the same config write that admits it. Ids are never reused or rolled back; a refused admission leaves its own, distinct id behind. An admission is proven only when its predecessor is proven settled — a reservation made by this build, or a predecessor with a durable receipt (or the same process's eligible settlement). Every other admission (startup re-drive, stale-starting relaunch, pre-upgrade entry, reawaken of an unproven or unsettled predecessor) persists taskAttemptUnproven: true; an unproven lineage never becomes proven again.
  2. Owner-written, immutable settlement receipts. When the owner of a proven attempt settles it — exactly where settleOwnedTaskAttempt accepts the settlement today — it first writes a receipt file named by that captured attempt id (atomic rename, never modified). Unowned attempts and unproven lineages write no receipt and stay indeterminate across processes. In-process ownership and settlement (ownedAttemptByTaskId, attemptSettlementByTaskId) keep exactly the semantics main has today for every admission, proven or not.
  3. Guarded consumption. A receipt lets a fresh process read terminal-no-report; replacement additionally requires a durable, monotonic claim: a config compare-and-set, in the same transaction space every publishing admission uses, that binds the retired attempt id to the exact workflow checkpoint (runId, stepId, inputHash) and is refused by any later admission (reawaken, reactivation, startup re-drive). A claim is never cleared. Read → dispose → claim → reserve is revalidated at each hop, and the claim happens at the single replacement chokepoint (reserveAgentTasks).

Guarantee delivered: a retired attempt is replaced at most once (a replacement child that is itself later retired is a new attempt and may be replaced again), and a resume/retry after a restart that replays successfully to a started/failed checkpoint whose child's proven owner settled it with a receipt and whose report is positively absent does replace that child (subject to the documented checkpoint-without-config-entry crash gap, which stays unresolved); reused reports and live children are untouched; no execution can be admitted or continued for a settled or claimed attempt; no receipt ever describes a child that a foreign process may still publish for. Not delivered (kept fail-closed, explicit reasons): attempts without a receipt — legacy children, owners that crashed after persisting interrupted but before the receipt was durable, unproven lineages (children found starting/running/awaiting_report at a restart and everything reawakened from them).

Net +450–620 product LoC (provisional until the admission audit in Change 1 and the TurnAdmissionToken wiring in Change 3a are mapped): attempt id and lineage (config fields, admission writers incl. startup re-drive rotation, the stale-starting marker, the exclusive launch CAS, proof evaluation at reawaken/reactivation, OwnedTaskAttempt, TaskLaunchPlan.sendAdmitted) ≈ 80–110; receipt module ≈ 70–90; receipt writes at the settlement producers incl. stop-record promise ownership and routing terminal failure through it ≈ 60–80; admission lifecycle (closing phase, AdmittedSend obligations, fence, stop-record pendingAdmissions/capturedTurns, receipt I/O outside the lock) ≈ 60–90; TurnAdmissionToken wiring through SendMessageInternalOptions, agentSession.sendMessage/resumeStream/sendQueuedMessages gates and MessageQueue entries ≈ 60–80; classifier branch ≈ 35; two-mode monotonic claim CAS + admission refusal ≈ 50–70; report-artifact attemptId ≈ 5; adapter/runner/types ≈ 40–60. Tests ≈ 900–1200 lines incl. one cross-process fixture. Deferred at 0 product LoC: operator-authorized replacement without a receipt, ownership handoff for startup re-driven tasks, checkpoint-retry eligibility / QuickJS normalization, in-process settlement gaps, removed-task tombstones, failure-tolerant parallel.


Evidence (read-only inspection, main 32834cca6)

  • inspectAttemptOutcome (src/node/services/taskService.ts:2294–2385) under workspaceEventLocks.withLock(taskId) (:2289, the publication serialization): persisted report first (:2302–2338; found → reported, unreadable → indeterminate, else reportPositivelyAbsent), stop latch (:2342), live registration (:2345), stream / active turn generation (:2353–2357), owned gate (:2361–2368), settlement (:2369–2377).
  • settleOwnedTaskAttempt (:2218–2233) accepts a settlement only for the attempt identity captured before the owner's awaited write and still current; otherwise it only announces. Callers: stop-record release (recheckWorkspaceStopRelease :2134–2146, gated on stopPersisted && turnSettled && executionSettled && cleanupInFlight === 0), reservation canceled (:4369) / failed (:4518), markTaskLaunchFailed (:4752), idle stop (:13238), failAgentTaskTerminally (:13407; callers :10106 hard timeout after stopStream + terminateAllDescendantAgentTasks, :12446 recovery limit, :13299 non-retryable stream error, :13340 context exceeded — all after the attempt's stream ended).
  • WorkspaceStopRecord (:790–812): cleanupInFlight is planned per cascade before any await and paid back only when the ORIGINAL promise settles, never by a timeout.
  • Admissions: reservation commit in createMany (beginOwnedTaskAttempt :4296), launch (startReservedAgentTask :4801, same attempt), reactivation (reactivateChildAgentTask :5886–5930: beginOwnedTaskAttempt then createWorkspaceTurn; on !execution.success it restores only in-memory state, and createWorkspaceTurn can throw after a successful send), reawaken (markInterruptedTaskRunning :12237–12296: beginOwnedTaskAttempt, then an editActiveWorkspaceEntry that sets running). Startup recovery (recoverInterruptedTasks :3175–3452) re-drives running/awaiting_report tasks via workspaceService.sendMessage / promptTaskForRequiredCompletionTool without beginOwnedTaskAttempt.
  • workspaceFileLocks is a process-local MutexMap (src/node/utils/concurrency/workspaceFileLocks.ts:23); subagent-failures.json read-modify-write (subagentFailureArtifacts.ts:118–135) is therefore not safe against a second process. Per-attempt files with atomic rename are.
  • The historical stuck children (255129c6d0, f1f83532d5, 8e1d8f616c, a69d2d6ee3) have failure artifacts but no execution mirror (taskExecutionId null: ordinary child launches never create workspace turns; only reactivation does, :5892). They are legacy for this design and remain indeterminate.
  • Config entry task fields are enumerated explicitly on write (src/node/config/index.ts:3551–3563, :3849–3861, :3925–3935); a downgraded build drops unknown fields.
  • WorkflowRunner.classifyPriorAttempt (WorkflowRunner.ts:2982–3033): reported → adopt, live → reattach, terminal-no-report → recordStartedAttemptFailed (inside classifyPriorAttempt, :3003–3008) + replace, indeterminate → WorkflowPriorAttemptUnresolvedError (:831–835, :3188–3209, lease released :893). replace is consumed on two paths — runOrResumeAgentStep (:2356restart() :2305–2311 clears taskIdreserveAgentTasks :2412) and pipeline startAgentStep (:1335:1375) — plus the failed-step re-run (:1242); all three end in reserveAgentTasks, the only creator of children. handleAgentWaitFailure (:3043–3123) replaces on terminal-no-report only for the exact restart sentinels. WorkflowTaskServiceAdapter.readSettledAgentResult (:186–195) passes requestingWorkspaceId: this.parentWorkspaceId. TaskAttemptOutcome (src/common/types/tasks.ts:11–16).
  • Reservation crash boundary is already closed. reserveAgentTasks appends a reserving event (:2815–2822, :3236–3249), then createMany invokes onTaskReserved before commitReservations (taskService.ts:4300–4310), and that callback durably writes the started step record and task event naming the new task id (WorkflowRunner.ts:2865–2894). A crash after the checkpoint but before the config commit leaves a started step naming a task with no config entry (indeterminate("no task record …"), the deferred tombstone gap — fail-closed, never a duplicate); a crash before the checkpoint leaves nothing to duplicate. The runner never scans config for workflowTask.runId/stepId; it only reads runStore.getStep (:1293, :1321, :1565, :1584, :1615, :1651).
  • Startup ordering. taskService.recoverInterruptedTasks is step 5 of startupCoreSteps (serviceContainer.ts:370–380) and completes before any listener binds (src/desktop/main.ts:769–790, src/cli/server.ts:147–170). Workflow runs are never marked interrupted at startup; a crashed running/backgrounded run keeps its persisted status (WorkflowRunStore.ts:327–333 reads status from events) until a client lists/subscribes, which triggers resumeCrashedRuns (WorkflowService.ts:225–240, :1128–1132) once the 30 s lease (WorkflowRunStore.ts:203, :1006–1013) is stale. Consequently a queued replacement child of a crashed running run is launched by the queue drain at startup (inactive-owner prepass returns null for active statuses, taskService.ts:1917–1919, src/common/types/workflow.ts:38–46) and the resumed run reattaches to it as live.
  • Stale-starting revert (taskService.ts:3198–3211): if (workspace.taskStatus !== "starting") return; workspace.taskStatus = isStreaming ? "running" : "queued"; and optionally clears taskPrompt. Nothing else changes, so a reverted entry is indistinguishable from a never-launched reservation — the queue drain would then launch it as if never admitted. Inactive-owner prepass (interruptTaskRecoveryForInactiveWorkflowOwner :2590–2599, applyInterruptedTaskStatus :2551–2571) interrupts on the fresh transaction read unconditionally (no queued re-check); it never begins ownership.
  • server.lock is an atomic-rename discovery record (serverLockfile.ts:38–103); the desktop keeps its backend running when another server holds it (src/desktop/main.ts:902–906) and the ACP in-process fallback runs without it (src/node/acp/serverConnection.ts:70–80). Nothing below depends on process exclusivity.
  • Existing spawn-based tests: src/node/services/projectService.test.ts:566–604 (Bun.spawn), streamManager.continuousCompaction.test.ts.

Proof obligations

# Obligation How it is met
P1 A receipt exists only when no execution — in this or any other process — can publish for the attempt anymore. Receipts are written inside the owner's settlement path, only when settleOwnedTaskAttempt would accept the settlement (captured attempt still current), only for attempts whose lineage is proven (P11, OwnedTaskAttempt.receiptEligible), and only by the producers whose last-publication boundary is established in the table below: execution settlement (turn and execution mirror settled), idle settlement (no execution exists), and pre-admission reservation/launch failure. Terminal failure with a live execution routes through execution settlement rather than writing at its call site. Every producer closes the attempt's admission (Change 3 linearization point) before its first awaited write, and the send fence observes that closure under the same lock, so no send can be admitted for an attempt after its receipt write begins. Unowned tasks and unproven lineages never produce receipts.
P11 Ownership can be acquired only over an attempt whose predecessor is proven settled; a foreign live execution can never be laundered into receipt authority. Lineage is proven by induction from the reservation: (a) the reservation commit is the first admission (queued entries are never admitted — admission persists starting first) and the exclusive queued → starting CAS hands that proof to exactly one launcher; (b) a reawaken/reactivation is proven iff the predecessor attempt (the entry's current taskAttemptId) has a durable receipt, or is owned by this process with an eligible, recorded settlement; (c) everything else — startup re-drive of running/awaiting_report, the stale-starting revert, a pre-upgrade entry without id, a reawaken whose predecessor is unsettled or unproven — persists taskAttemptUnproven: true, which every successor inherits. receiptEligible is fixed at admission from that evidence and travels with the owned attempt; persistOwnedAttemptSettlement writes nothing when it is false. In-process ownership/settlement is granted exactly as on main regardless of proof, so same-process behavior is unchanged; only cross-process authority is gated.
P2 A receipt names the attempt that settled, never "whichever attempt is current". The id travels in OwnedTaskAttempt.attemptId, captured before awaits at each site (existing pattern :4744–4752, :13399–13407, stop record capture). The receipt path is derived from that captured id; config is not consulted when writing.
P3 Identity is never reused. Fresh opaque id per admission, persisted before/with admission; refused or failed admissions keep their id (no rollback); pre-upgrade entries get an id at their next admission; a downgrade→upgrade cycle assigns a new id rather than re-deriving one.
P4 A stale receipt cannot authorize replacement of a live or newer attempt. Classification requires receipt.attemptId === entry.taskAttemptId after a strict config read, plus all existing in-process liveness checks (including admitted-but-not-yet-visible sends); the claim re-checks the same conditions under the task's event lock and commits a config CAS that reawaken/reactivation refuse. A receipt's evidence stays true afterwards because settlement closes admission for that id (Change 3).
P5 Classification → replacement is not a TOCTOU. Disposition (recordStartedAttemptFailed, inside classifyPriorAttempt) is a checkpoint fact, not a replacement; replacement happens only in reserveAgentTasks, which performs claimRetiredAttempt(taskId, attemptId, { runId, stepId, inputHash, mode }) immediately before createAgentTasks. The claim is durable, monotonic (never cleared), idempotent for the same { runId, stepId, inputHash, childTaskId, attemptId } under any lease holder, and refused for any other checkpoint. Every publishing admission's mutator refuses when taskAttemptRetiredBy is set (P9), so no admission can start work for a claimed attempt; a reawaken that commits between disposition and claim makes the claim fail and the run unresolved with the failed checkpoint intact.
P6 Fail closed on every read/write failure. Strict config reads (loadConfigOrDefault({ throwOnError: true }), config/index.ts:1450) → indeterminate; receipt unreadableindeterminate; claim CAS failure → WorkflowPriorAttemptUnresolvedError; receipt write failure → no durable settlement (logged), same-process settlement still recorded so same-process behavior is unchanged.
P7 Teardown stays bounded for callers, never for the write's ownership. Every receipt write runs outside the task event lock as an owed cleanup bound to the captured attempt (idle/terminal paths) or stop record (stop path), so inspectAttemptOutcome can always observe closing/cleanup-pending. The stop-path receipt write is an owed cleanup on the captured stop record, paid back only when its own promise settles (the existing cleanupInFlight rule, :793–799). A confirmed rejection pays it back and same-process settlement proceeds with durable: false. A still-pending write keeps the latch retained: callers' waits stay bounded by the existing stop aggregate deadline (Layer 2: the latch is retained, the caller returns), inspectAttemptOutcome reports cleanup-pending, and a late completion releases the latch exactly once. No deadline ever settles, cancels or double-counts the write.
P9 Admission and claim are mutually exclusive in one shared transaction. Both run as editConfig mutators on the transaction's fresh read; editConfig serializes writers in-process (FIFO) and across processes (each save re-verifies this process's hold of the project registration lock, config/index.ts:2872–2884). The claim mutator requires a terminal status, taskAttemptId === expected, the child's parent/workflow association, and an absent or identical taskAttemptRetiredBy; every publishing admission's mutator requires taskAttemptId === expectedPrevious and taskAttemptRetiredBy == null, then publishes its fresh id; prepared sends re-check at dispatch. Whichever commits first makes the other fail. Guarantee boundary: writers running this build or newer. Mixed-version concurrent writers on one root are outside the guarantee.
P10 No replacement bypasses the claim, and no crash duplicates a replacement. Replacement children are reserved only in reserveAgentTasks, which claims the prior child's attempt for started and failed checkpoints alike before creating tasks; the Stop-drain disposition records failed but never reserves. What the existing order checkpoint-before-commit (taskService.ts:4300–4310, the createMany path used by the real adapter; the adapter's createMany == null single-task fallback WorkflowTaskServiceAdapter.ts:301–306 is outside this proof and is asserted unreachable with the real service) proves is at-most-once committed replacement reservation per retired attempt (the claim identity names the attempt; a step legitimately retried several times retires several attempts) — not unconditional exactly-once recovery: a crash before the checkpoint leaves the failed/started record naming the old child (the idempotent claim is re-run); a crash after the checkpoint names the new child (launched by startup as queued → live); a crash between checkpoint and config commit leaves a started step naming a child with no entry, which reads indeterminate and stays unresolved indefinitely until the deferred tombstone follow-up.
P8 Cross-process durability. One file per attempt, written by temp+rename; readers parse a versioned JSON with a strict schema; concurrent writers of different attempts never touch the same file; a second write of the same path is a no-op if the file exists.

Residual, stated in code comments and tests: an attempt whose owner crashed after persisting interrupted but before its receipt was durable is indeterminate forever (until the deferred operator override); an unproven lineage (startup re-driven, stale-starting relaunch, or reawakened from either) never produces a receipt, so its terminal settlement is recoverable only in the process that owns it (as on main), not across a further restart.

Receipt producers and their last possible report-publication boundary

Only producers whose boundary is established from the code are kept; everything else stays without a receipt (fail closed).

Producer Boundary after which no report can be published for the attempt Status
Execution settlement through the stop-record release (recheckWorkspaceStopRelease :2134–2146): today stopPersisted && turnSettled && executionSettled && cleanupInFlight === 0; after Change 3a stopPersisted && capturedTurns.size === 0 && pendingAdmissions.size === 0 && executionSettled && cleanupInFlight === 0. Used by user Stop, terminateAllDescendantAgentTasks, and — new — every failAgentTaskTerminally call while the task still has a live turn/registration/stream (the terminal-failure path opens a stop record instead of settling immediately). Turn generation ended and execution mirror settled. Report publication runs inside stream-end handling (handleStreamEnd → finalizeAgentTaskReport), which precedes turn settlement (onWorkspaceTurnSettled, workspaceService.ts:13375recordWorkspaceTurnSettled :2184). Layer 2's monotonicity note (:8735–8739: an interrupted task can still be streaming while stream-end persists agent_report) is exactly why terminal failure must not write at the call site. Kept
Idle terminal settlement: idle stop (stopDescendantAgentTaskUnderLifecycleLock :13229–13238) and failAgentTaskTerminally when no live turn generation, registration or stream exists for the task. No execution exists; the checks run under the task's event lock (implementation moves them there if any site evaluates them outside it). Kept
Reservation canceled / failed (:4361–4369, :4501–4518). Before any launch. Kept
Launch failed before this process's own admission (markTaskLaunchFailed, any call site). Positive captured evidence, not a call-site whitelist: the failing plan's attempt is owned by this process (ownedAttemptByTaskId.get(id)?.attemptId === plan.attemptId) and the plan's sendAdmitted flag — set at the provider-start fence immediately before the send is admitted — is still false. Sites acting on unowned entries (:3210 startup stale-starting, :3666 desktop-recovery admission, queue-drain validation before the launch CAS :11971–12010) fail the ownership test and write nothing; :4723/:4786 write a receipt only when the flag proves no admission. Kept under the captured-evidence rule
reactivation-refused createWorkspaceTurn returns Err at many points after validation (workspaceTurnManager.ts:1084, :1111, :1219, :1239, :1351), so Err does not prove "no admission". Dropped

Change 1 — Immutable attempt identity

  • WorkspaceConfigEntry.taskAttemptId?: string and taskAttemptRetiredBy?: { runId: string; stepId: string; inputHash: string; childTaskId: string; attemptId: string; mode: "no-report" | "retire-reported"; at: string } (src/node/config/index.ts type near :197, parse site near :1104, the three explicit field lists). Both optional; absent on legacy entries.
  • Third optional field taskAttemptUnproven?: true (same three field lists): present when the attempt's lineage is not proven (P11). Written together with taskAttemptId by the admission that rotates the id; omitted by the reservation commit and by proven reawaken/reactivation; inherited (copied) by every admission that does not itself prove the lineage. Never cleared except by a new reservation (a new task).
  • OwnedTaskAttempt (taskService.ts:443–452) gains readonly attemptId: string | undefined and readonly receiptEligible: boolean; beginOwnedTaskAttempt(taskId, source, { abortSignal?, attemptId, receiptEligible }). receiptEligible is decided once, at admission, from the evidence in P11; it is never recomputed later.
  • Proof rule at reawaken/reactivation (one helper evaluateAttemptLineage(taskId, entry): Promise<{ proven: boolean; reason: string }>, evaluated under the task's event lock before the admission CAS): proven iff entry.taskAttemptId != null && entry.taskAttemptUnproven !== true and either (i) ownedAttemptByTaskId.get(taskId) has that attemptId, receiptEligible === true, and attemptSettlementByTaskId records its settlement — if the owned attempt is still settling (stop latch retained), wait with the existing bounded waitForAttemptSettlement first, then re-evaluate; or (ii) readSubagentAttemptSettlementReceiptStrict(parentDir, taskId, entry.taskAttemptId) is found (unreadable/not_found → not proven). The bounded waitForAttemptSettlement runs outside the task event lock (it waits for publication/settlement work that needs the lock); the lock is then reacquired and the proof re-evaluated. The admission CAS publishes taskAttemptId = newId and decides the marker from the fresh transaction row: proven = precomputedProven && ws.taskAttemptUnproven !== true (the marker is only ever added on a given task, so a marker that appeared after the snapshot can only downgrade). The mutator records its committed result, and beginOwnedTaskAttempt(…, { receiptEligible }) is called with that committed value — never with a pre-await snapshot. Unproven admissions proceed exactly as on main (owned in memory; no receipts) and log the reason once.
  • Id generation: att_ + 16 hex chars from crypto.randomBytes (src/node/utils/… helper next to existing id helpers); assert(/^att_[0-9a-f]{16}$/.test(id)) at every write and read.
  • Writers (each inside the existing config mutation that performs the admission, so id and status change atomically):
    • reservation commit in createMany (the editConfig that persists reserved plans, near :4296): taskAttemptId = newId, no taskAttemptUnproven; the OwnedTaskAttempt created at :4296 carries it with receiptEligible: true (the reservation is the first admission by construction).
    • startReservedAgentTask (:4758–5072): keeps the reservation's id and eligibility; if the entry has none (pre-upgrade queued/starting entry), assign one in its existing status write with taskAttemptUnproven: true (an old build's stale-starting revert may have reverted an admitted entry to queued without any marker) and carry it into beginOwnedTaskAttempt("launch", { receiptEligible: false }).
    • markInterruptedTaskRunning (:12237–12296): strict pre-read refuses when taskAttemptRetiredBy != null (returns false, before beginOwnedTaskAttempt); evaluateAttemptLineage decides proven; the mutator re-checks (if (ws.taskAttemptRetiredBy != null || ws.taskAttemptId !== previous) return;) and sets taskAttemptId = newId, taskAttemptUnproven per the proof, with running. On mutator refusal, restore the previous in-memory ownership/settlement exactly as reactivateChildAgentTask does (:5911–5925).
    • reactivateChildAgentTask (:5886–5930): before createWorkspaceTurn, evaluateAttemptLineage, then an editWorkspaceEntry CAS (taskAttemptId === previous && taskAttemptRetiredBy == null) publishes taskAttemptId = newId and taskAttemptUnproven per the proof (same lifecycle lock), then beginOwnedTaskAttempt("reactivation", { attemptId, receiptEligible: proven }). Boundary: if the CAS fails (claimed or id changed), nothing was published — restore speculative memory as today (:5911–5925) and return the refusal. Once the CAS committed, the new identity is kept in both config and memory whatever createWorkspaceTurn returns or throws: the old attempt is never restored after its successor is published. A refused/failed reactivation therefore leaves an owned, unsettled attempt (indeterminate, reason "owned attempt without settlement evidence"); a subsequent Stop settles it through the idle or execution-settlement producer and writes its receipt. The existing in-memory restore block is reduced to the pre-commit case.
    • Launch ownership for plans this process did not reserve (queue drain maybeStartQueuedTasksFromReservations, :12082–12084, which today sets starting unconditionally): the transition becomes a CAS taskStatus === "queued" && taskAttemptRetiredBy == null → starting that also rotates taskAttemptId when ownedAttemptByTaskId.get(id)?.attemptId !== entry.taskAttemptId and copies taskAttemptUnproven unchanged; the process whose CAS commits is the single owner (beginOwnedTaskAttempt("launch", { attemptId, receiptEligible }) with receiptEligible = ws.taskAttemptUnproven !== true read from the fresh row inside the mutator, not from the drain's snapshot — a stale-starting revert by another process can add the marker without changing the id), the loser skips the plan. A queued entry without the marker has by definition never been admitted (admission persists starting first, and the only path back to queued sets the marker below), so this handoff is exclusive and proven by construction.
    • Stale-starting revert (:3198–3211): the mutator additionally sets taskAttemptUnproven = true when it writes queued (or running). A starting entry found at startup may already have an admitted execution in another process; the marker makes the relaunch — which otherwise looks exactly like a never-launched reservation — an unproven lineage, so the drain's launcher owns it in memory as today but is never receipt-eligible. Entries a process finds in running or awaiting_report may likewise have an admitted execution elsewhere: the existing re-drive keeps running unowned — no beginOwnedTaskAttempt, no receipts — and marks the lineage unproven (below), so no receipt can ever describe a child that another process may still publish for, and nothing reawakened from that lineage can produce one either.
    • Startup re-drive (recoverInterruptedTasks: guidance replay :3374, restart nudge :3417, promptTaskForRequiredCompletionTool({ reason: "startup" }) :3317, dispatchPendingCompactionFollowUp :3313/:3358): before the send, an editWorkspaceEntry CAS rotates taskAttemptId and sets taskAttemptUnproven = true (refusing when taskAttemptRetiredBy != null, in which case the task is skipped and logged). The task stays unowned (no beginOwnedTaskAttempt, no receipts), any receipt of the previous attempt can no longer match the execution this process starts, and every later reawaken of this child inherits the unproven marker.
    • Within-owner continuations of the same attempt do not rotate: in-owner recovery prompts (promptTaskForRequiredCompletionTool :12466/:12539 from :13127, :13366, :13707, :14390), parent guidance queued into a live child (:6192, :6993), best-of/continuation kickoffs of an owned attempt (:9460:10020, :12823:12876, :13848). Rule, enforced by one helper admitTaskSend(taskId, { newAttempt }) used at every send site: if the attempt is admission-openownedAttemptByTaskId.get(taskId)?.attemptId === entry.taskAttemptId, and attemptSettlementByTaskId has no entry for it (neither closing nor settled, Change 3), and no workspaceStopRecords entry exists for the workspace, and taskAttemptRetiredBy == null — the send continues the same attempt (no rotation) and is registered as an AdmittedSend obligation bound to that attempt until discharged (Change 3); if the attempt is owned but settled or settling, the send is refused (Err("attempt settled"), logged; the caller drops it — an intentional continuation after settlement is a reawaken and must go through the fresh-id CAS); otherwise the helper performs the rotation CAS (newAttempt: { own: true, receiptEligible } also begins ownership; newAttempt: { own: false } is the unowned startup re-drive, which rotates and marks but never owns), refusing when claimed. Settlement therefore closes an attempt to further sends, so no continuation can dispatch under an id that a receipt already describes (P1/P4). Uses existing settlement/stop state only; no new persisted field. Enumerate every workspaceService.sendMessage / prompt call on a task workspace in taskService.ts (the line list above is the seed), route each through the helper, record the classification in a comment at each site (test 1b covers each).
    • Dispatch-time identity check. A committed CAS does not cancel a send that was already prepared. Every task send carries expectedAttemptId; the admission fence Layer 2 added before the provider start (the check "at physical turn admission, immediately before sendMessage" used by startReservedAgentTask, and the AgentSession/StreamManager provider-start fence) re-reads the entry under the task's event lock and aborts the send when taskAttemptId !== expectedAttemptId or taskAttemptRetiredBy != null; the authoritative in-flight check is the existing admissionStale probe (SendMessageInternalOptions, evaluated synchronously before coordinator.prepare and at dequeue) supplied by admitTaskSend as () => !attemptAdmissionOpen(taskId, expectedAttemptId) — a continuation prepared before a Stop is refused at the session gate, never dispatched after the receipt (Change 3). Implementation verifies the exact seam (the fence added for TaskLaunchPlan.abortSignal) and reuses it rather than adding a second fence.
  • No other code reads taskAttemptId for behavior; it is identity only.

Change 2 — Receipt module (src/node/services/subagentAttemptSettlements.ts, new)

  • Path: sessions/<ownerWorkspaceId>/subagent-attempt-settlements/<encodeURIComponent(taskId)>/<attemptId>.json, written into the parent and every ancestor session dir (same fan-out as failure artifacts; ancestors resolved from the captured entry, not re-read).
  • Record: { version: 1, taskId, attemptId, parentWorkspaceId, source, settledAt } with source ∈ { "execution-settled", "idle-settled", "launch-failed", "reservation-canceled", "reservation-failed" } (terminal failures appear as execution-settled or idle-settled depending on the producer that wrote them; the failure artifact keeps the error detail).
  • writeSubagentAttemptSettlementReceipt(params): Promise<Result<void, string>> — temp file + rename; if the target exists, verify equal attemptId and return Ok (idempotent). Never throws; it settles only when the OS write settles (no internal timeout), so ownership of the write is never abandoned.
  • readSubagentAttemptSettlementReceiptStrict(ownerDir, taskId, attemptId): Promise<{ kind: "found"; receipt } | { kind: "not_found" } | { kind: "unreadable"; error }> — ENOENT → not_found; any other error or schema failure → unreadable.
  • Deletion: none (immutable; bounded by attempts that ended without a report, like the in-memory receipts).

Change 3 — Admission lifecycle and receipt writes (taskService.ts, workspaceService.ts, agentSession.ts, messageQueue.ts)

private async persistOwnedAttemptSettlement(taskId, attempt: OwnedTaskAttempt | undefined, source): Promise<boolean> — returns false without writing when attempt == null, attempt.attemptId == null, attempt.receiptEligible === false (unproven lineage, P11), or this.ownedAttemptByTaskId.get(taskId) !== attempt (the same guard settleOwnedTaskAttempt applies, evaluated before the write); otherwise writes the receipt for attempt.attemptId and returns whether it became durable. Then the caller calls settleOwnedTaskAttempt as today (in-memory settlement is recorded even if the write failed; the log line distinguishes durable: true|false). It is called only from the producers in 3b, which are enabled only after gate G1 (3a).

Change 3a — Admission lifecycle contract (prerequisite; gate G1 must be green before any receipt producer is enabled)

Linearization point — admission closes before the first awaited receipt/config operation (P1/P4). Today no producer marks an attempt as settling before its awaited write (OwnedTaskAttempt is { generation, source, abortSignal? }, immutable; attemptSettlementByTaskId is populated only after editWorkspaceEntry/cleanup awaits in all six callers), and the launch fence at startReservedAgentTask:5028–5045 holds no lock, so a send prepared before a Stop could dispatch during the receipt write. The fix reuses the existing settlement map and stop records and adds one in-memory obligation set plus one explicitly planned integration: a TurnAdmissionToken carried in the send options through WorkspaceService, AgentSession and the MessageQueue (the WorkspaceService-facing SendMessageInternalOptions does not expose onTurnAdmissionCommitted today — that callback is AgentSession-internal and releases WorkspaceService's preflight reservation; the token is forwarded alongside it and composes with it, never replaces it). Changes 1–2 and 3a ship and are verified together (G1, receipt-independent — see Quality gates) before 3b–5 enable receipts and claims. A partially wired token fails closed during development (an obligation that can never be dispositioned keeps the stop latch retained → cleanup-pending, visible, never a receipt) — that is a failing G1, not a passing one.

  • attemptSettlementByTaskId entries become { attemptId: string | undefined; attempt?: OwnedTaskAttempt; phase: "closing" | "settled"; source }. closeAttemptAdmission(taskId, { attemptId, attempt }, source) records the closing entry synchronously, under workspaceEventLocks.withLock(taskId), before the producer's first await, with attemptId captured from the fresh config row inside the producer's updater (so unowned attempts are closed for their own id too; attempt only when owned); settleOwnedTaskAttempt upgrades the same entry to settled (its existing owned-guard unchanged). beginOwnedTaskAttempt/a new-attempt admission deletes the entry (a fresh id reopens admission). An entry in any phase is never removed by a failed write: admission stays closed; intentional continuation is a reawaken (fresh id). Every consumer that treated settlement-map membership as settlement now requires phase === "settled": the owned path of inspectAttemptOutcome (:2369–2377, closingcleanup-pending), waitForAttemptSettlement, evaluateAttemptLineage (i), the claim's same-process evidence, and handleAgentWaitFailure's consumers through the adapter.
  • Send obligations with immutable identity. admittedSendsByTaskId: Map<taskId, Set<AdmittedSend>>, AdmittedSend = { readonly attemptId: string; readonly attempt?: OwnedTaskAttempt; state: "pending" | "enqueued" | "admitted" | "discharged"; turnId?: TurnId }attemptId is the captured config id the send was admitted under (non-optional, also for unowned startup sends); attempt only for owned admissions (receipt authority). The obligation is created by the fence before dispatch and dispositioned only through its TurnAdmissionToken; the send call's return value is never used as evidence. Discharge updates the obligation object and the record sets that captured it by reference, never whichever record currently occupies the task's map slot; an obligation admitted under attempt X can never enter, or be removed from, a successor attempt's record.
  • TurnAdmissionToken (new field on the WorkspaceService-facing SendMessageInternalOptions, taskWorkspaceSeam.ts:330–370, forwarded into AgentSession's internal options agentSession.ts:745–831 and stored on the MessageQueue entry exactly like the existing admissionStale): { admissionStale(): boolean; onAdmitted(turnId: TurnId): void; onDisposed(kind: "no-work" | "refused" | "canceled-before-admission"): void }. onAdmitted is idempotent per turnId (a dequeued item is prepared at :10128 and then adopted by sendMessage(…, { turnReservation }) at :10188 — the second call must not create a second admission); onDisposed is idempotent and ignored once admitted. Fired at the first actual preparation admission of every path, never inferred later:
Seam (verified) Token event Obligation
workspaceService.sendMessage pre-queue/pre-session Err sites (:11696–11844, :11864, :11939/:11971, :12005–12010, :12051–12063, :12188–12190) and agentSession.sendMessage pre-prepare refusals (:3338, :3669/:4293, :4300, the admissionStale gate :4791–4805, prepare rejected/deferred :4834–4842) onDisposed("refused") discharged (never admitted)
dedupe / restore returns (:11759, :11763) and resumeStream Ok({ started: false }) (busy :4970, closing :5004, prepare rejected :5039) onDisposed("no-work") discharged — no work exists for this token; the deduped-into item keeps its own token
session.queueMessage (:12127–12159) (none yet; entry stores the token) enqueued — stays until its own dequeue
dequeue sendQueuedMessages (:10097–10201): before coordinator.prepare at :10128 evaluate the item's admissionStale (added at that gate if absent) → stale ⇒ item dropped, onDisposed("refused"); else prepare admitted ⇒ onAdmitted(turnId) here, before the sendMessage(…, { turnReservation }) re-entry at :10188 as stated admitted from :10128 on — an early failure inside the :10188 re-entry is not "never admitted"; its turn settles through the preparation-failure path
direct coordinator.prepare admitted (agentSession.ts:4817; existing onTurnAdmissionCommitted :4846 keeps releasing WorkspaceService's preflight reservation) and resumeStream prepare admitted (:5030) onAdmitted(turnId) with the admitted id passed explicitly admitted
streamWithHistory failures after admission (:7480, :7483, :7736, :7813/:7879, resume :5071) (none) stays admitted; discharged only when that turn settles (settlePreparationFailure → turn completion → recordWorkspaceTurnSettled; implementation verifies the turn-settled event fires for failed-start turns, otherwise extends that seam)
cancellation (onCanceled, cancelSignal, clearQueue via Stop taskService.ts:7227/:7726 → workspaceService.ts:2078, interruptStream :12879 → agentSession.ts:10028) onDisposed("canceled-before-admission") only if not yet admitted pending/enqueued → discharged; admitted (PREPARING or later) → retained until the correlated turn settles — cancellation requests termination, it does not prove settlement
turn settlement recordWorkspaceTurnSettled(turnId) (TaskService-internal) admitted with that turnId → discharged; removed from every capturing record's capturedTurns
supersession (a new taskAttemptId published) (none) existing obligations keep their attemptId; admitted ones are retained until settlement; enqueued ones are refused at dequeue by their own admissionStale

token.admissionStale = () => token.state === "discharged" || !attemptAdmissionOpen(taskId, token.attemptId) || (token.attempt != null && ownedAttemptByTaskId.get(taskId) !== token.attempt) reads TaskService in-memory state only and is the refusal authority: evaluated synchronously at the enqueue block (:12051, :12188), at the session gate (:4791) and at dequeue (:10128), it returns Err before any coordinator state changes. The common predicate attemptAdmissionOpen(taskId, id) (send authorization, no ownership) requires: currentAttemptIdByTaskId.get(taskId) === id — an in-memory mirror of the task's current taskAttemptId, updated synchronously by every local rotation (reservation commit, drain launch CAS, reawaken, reactivation, startup re-drive rotation, owned or not), so any later local rotation revokes every older pending/enqueued token by construction, including unowned startup generations (startup Y in preflight → startup Z rotates → Y reaches prepare is refused); no closure entry for id; no workspaceStopRecords entry. Ownership is checked separately and only where it applies: a same-attempt continuation requires ownedAttemptByTaskId.get(taskId)?.attemptId === id at the fence and, through token.attempt, at every later gate; a new owned attempt passes the common predicate for the just-published Y first and then, in the same synchronous block, installs ownership (beginOwnedTaskAttempt(Y)) and registers its obligation with token.attempt set — so owned X settled → CAS publishes Y → admission of Y succeeds (the previous owned X never blocks Y); a startup re-drive passes the common predicate and registers an unowned obligation (token.attempt undefined), never acquiring ownership to satisfy any check. beginOwnedTaskAttempt(newId) deletes the settlement entry only when entry.attemptId !== newId: a closure recorded for Y between the CAS and the block makes the block refuse and is never erased. A disposed token is terminal: onDisposed sets state = "discharged" and the token can never admit afterwards even if its attempt stays open. admitted tokens are unaffected by rotation (they are retained until their turn settles). So Y passes the fence → awaits → Stop closes Y → prepare and Y → Z supersedes → Y prepares are both refused. There is no "send settled + workspace idle" discharge and no inference from Ok/Err.

  • Stop record. WorkspaceStopRecord gains attemptId (the captured attempt's id; for an unowned attempt the task index's current id at capture), pendingAdmissions: Set<AdmittedSend> (the exact obligation objects for that attemptId, captured by reference in Phase A) and capturedTurns: Set<turnId> (replacing the single capturedTurn; initialized from the current turn). Release (recheckWorkspaceStopRelease) requires stopPersisted && capturedTurns.size === 0 && pendingAdmissions.size === 0 && executionSettled && cleanupInFlight === 0. When a pending admission's turn becomes visible, the record adds it to capturedTurns and owes one more cleanup (cleanupInFlight += 1) for a second aiService.stopStream(taskId) so the late turn is actually stopped; its settlement removes it. A captured turn settling while another admission is pending therefore cannot release the record (the round-10 counterexample: T1 settles, receipt, then admitted T2 starts).
  • Per producer: idle stop (releaseSharedDesktopTaskOnUserStop, already under the event lock :2692–2695; idleness verified inside the updater :13221–13227) and failAgentTaskTerminally (under the event lock in every streaming caller :2704–2707, :10077; the startup caller :3317 acts on an unowned task and writes no receipt) treat any pending admission as live (→ stop-record route, never an idle receipt) and otherwise call closeAttemptAdmission right after their idleness decision, before editWorkspaceEntry. The stop cascade's Phase A record creation (beginWorkspaceStop :2017–2050, synchronous under the global mutex, ownedAttempt captured) is the closure for the execution-settlement producer: the admission-open predicate treats an existing workspaceStopRecords entry as closed. Reservation canceled/failed: the plan's abortSignal is aborted before the write and the launch fence's abortSignal.aborted check (:5028–5033, the existing linearization comment) refuses the launch — no new marker. markTaskLaunchFailed: takes the event lock, calls closeAttemptAdmission, and writes a receipt only if plan.sendAdmitted === false and no admission is pending for the attempt, decided inside that lock.
  • Receipt I/O runs outside the event lock on every path. Idle/terminal producers, under the lock: idleness decision → closeAttemptAdmission → the existing editWorkspaceEntry → register the receipt write as an owed cleanup bound to the captured attempt → release the lock. The write then runs outside; on completion (fulfilled or rejected) settleOwnedTaskAttempt(taskId, attempt, source) upgrades the entry to settled against the captured identity (durable: true|false). While the write is pending, inspectAttemptOutcome (which takes the same lock) sees closing and returns cleanup-pending, and a fence call returns a refusal immediately rather than queueing on the lock. The stop path keeps P7's owed-cleanup form.
  • Fence (admitTaskSend, Change 1) runs under workspaceEventLocks.withLock(taskId) up to dispatch. Its awaited step is the strict config read (continue same attempt) or the rotation CAS that publishes the fresh id (new attempt: reawaken, reactivation, drain launch, startup re-drive). Then, synchronously: expectedId (X for a continuation, the just-published Y) equals the row's taskAttemptId; taskAttemptRetiredBy == null; the common attemptAdmissionOpen(taskId, expectedId) (generation current, no closure entry for expectedId, no workspaceStopRecords entry); for a continuation only, ownedAttemptByTaskId.get(taskId)?.attemptId === expectedId. Only then, still in the same synchronous block: owned new attempts beginOwnedTaskAttempt(expectedId, { receiptEligible }); register AdmittedSend { attemptId: expectedId, attempt?, state: "pending" } (attempt set for owned admissions, undefined for startup re-drives); set plan.sendAdmitted = true; call sendMessage/resumeStream with the obligation's TurnAdmissionToken. The pre-dispatch checks are an early exit; the authoritative refusal is the token's admissionStale, evaluated by the session synchronously before every coordinator.prepare (direct :4791, dequeue :10128, resume) so the asynchronous work inside sendMessage (compaction admission, pricing gate, preflight, history publish — workspaceService.ts:11860–11982, agentSession.ts:4321–4370) cannot admit a stale identity: Y passes the fence → awaits → Stop closes Y → prepare is refused at :4791. A refusal after a committed CAS leaves the fresh id published (P3: no rollback), the owned-but-never-admitted attempt readable as indeterminate (owned, unsettled) until a Stop settles it with its receipt, and returns Err. Send authorization is separate from receipt authority: startup re-drives pass the new-attempt branch unowned (fresh marked id, no beginOwnedTaskAttempt, no receipts), exactly the behavior Change 1 preserves, but they cannot send through a concurrent Stop or a closed attempt; owned branches acquire receiptEligible only per P11. closeAttemptAdmission records the closed attemptId also for unowned attempts (captured from the fresh row inside the producer's updater), so an unowned startup attempt that is Stopped is closed for its own id as well.
  • Queued input. Dequeue does not re-enter workspaceService.sendMessage (sendQueuedMessages, agentSession.ts:10097–10201, calls coordinator.prepare at :10128 and agentSession.sendMessage at :10188 directly), so the queued item's own token is the fence there (table above), together with Stop semantics: the task stop cascade runs runWorkspaceStopCleanup with clearQueue: true (taskService.ts:7227, :7726workspaceService.clearQueue, :2078), sendQueuedMessages freezes while a stop latch is held (agentSession.ts:10109), and interruptStream restores the queue to input (workspaceService.ts:12879clearQueue, agentSession.ts:10028). Cleared, not-yet-admitted items are dispositioned canceled-before-admission. The idle/terminal producers add workspaceService.clearQueue(taskId) to their closure step (the terminal-failure stop-record route already gets it from Phase B), so no deferred dispatch can outlive a closure; a bare aiService.stopStream is never used as a settlement producer (it does not touch the queue).

Change 3b — Receipt writes at the producers (enabled only after G1)

Ordering at every producer follows 3a: closure (closeAttemptAdmission, clearQueue where applicable) → existing config write → release the event lockpersistOwnedAttemptSettlement as an owed cleanup bound to the captured attempt → settleOwnedTaskAttempt (upgrade to settled, durable: true|false).


Generated with xum • Model: coder:openai/gpt-6-astra • Thinking: xhigh • Cost: $533.73

@ThomasK33

Copy link
Copy Markdown
Member Author

Approved implementation plan (2/2)

Full-plan SHA-256: b3c920d8666fc96eebcc9bbc64d21a6192bc8fd2f1c0e5970a0ff0be9825c029. Records G1 and future G2; review only G1 here.

Verbatim plan, part 2
  • Reservation canceled (:4361–4369), reservation failed (:4501–4518): the plan's abortSignal is the closure (launch fence :5028–5033); receipt write then settleOwnedTaskAttempt. Idle stop (releaseSharedDesktopTaskOnUserStop :13203–13242): closure inside the existing updater, then the ordering above. markTaskLaunchFailed (:4733–4756): closure under the event lock; receipt only when the plan is owned by this process, plan.sendAdmitted === false (TaskLaunchPlan.sendAdmitted, flipped by the fence right before dispatch; plans rebuilt from config carry no flag and never qualify) and no obligation is pending for the attempt.
  • failAgentTaskTerminally (:13379–13407): under the task's event lock decide before persisting interrupted: no live turn generation, registration, stream or pending obligation → idle producer (closure, clearQueue, config write, then the ordering above); otherwise open a stop record for the captured attempt (same Phase A capture the Stop cascade performs: capturedTurns, pendingAdmissions, capturedExecutionId, stopPersisted = true, Phase B with clearQueue: true) and let the execution-settlement producer write the receipt when the record releases. In-memory settlement moves to that release for this case (today it is recorded immediately at :13407; waitForAttemptSettlement already treats the latch as cleanup-pending, so consumers observe the same bounded wait they do for a Stop).
  • Stop-record release (recheckWorkspaceStopRelease :2134–2146): when the 3a release predicate holds (stopPersisted && capturedTurns.size === 0 && pendingAdmissions.size === 0 && executionSettled && cleanupInFlight === 0) and record.receipt == null, set record.receipt = { state: "pending", promise }, cleanupInFlight += 1, and start persistOwnedAttemptSettlement(id, record.ownedAttempt, "execution-settled") bound to that record (captured by reference, not looked up again). When the promise settles (fulfilled or rejected, whenever that happens): cleanupInFlight -= 1, record.receipt.state = "done", recheck → latch releases once and in-memory settlement is recorded. A record whose receipt is pending/done never schedules another write; a replacement record (new cascade after release) has its own field. No deadline touches this promise: while it is pending the latch is retained (fail-closed cleanup-pending), callers are bounded by the existing stop aggregate deadline, and the promise stays owned by the record through service disposal (disposal awaits or detaches it exactly as it does other owed cleanups today — verify and mirror that path).
  • reactivateChildAgentTask refusal (Change 1).

Change 4 — Classifier (inspectAttemptOutcome, unowned gate :2361–2368)

if (owned == null) {
  if (entry != null && status === "interrupted" && reportPositivelyAbsent) {
    const attemptId = entry.workspace.taskAttemptId;
    if (attemptId == null) return indeterminate("interrupted prior attempt without an attempt id (legacy)");
    const receipt = await readSubagentAttemptSettlementReceiptStrict(reportOwnerDir, taskId, attemptId);
    if (receipt.kind === "unreadable") return indeterminate(`settlement receipt unreadable: ${receipt.error}`);
    if (receipt.kind === "not_found") return indeterminate(`no settlement receipt for attempt ${attemptId}`);
    const latest = findWorkspaceEntry(this.config.loadConfigOrDefault({ throwOnError: true }), taskId)?.workspace; // throw → caught → indeterminate
    if (latest?.taskStatus !== "interrupted" || latest.taskAttemptId !== attemptId) return indeterminate("attempt changed while its receipt was being read");
    return { kind: "terminal-no-report", attemptId };
  }
  return indeterminate(/* existing reasons */);
}
  • TaskAttemptOutcome gains identity on every variant: { kind: "id"; attemptId: string } (strict read found the entry with an id — for terminal-no-report it equals the receipt/owned id by the checks above), { kind: "legacy" } (strict read found an existing entry without taskAttemptId), or { kind: "unknown" } (no entry, or the strict read failed). Existing consumers ignore it; the runner uses it to distinguish a positively identified legacy child from an unidentified one (P6). The indeterminate variant additionally gains a typed code ("legacy-no-attempt-id" | "no-task-record" | "unowned-no-receipt" | "receipt-unreadable" | "report-unreadable" | "config-unreadable" | "owned-unsettled" | "identity-changed") next to the human reason, so consumers branch on codes, not strings.
  • The owned backstop (:2382–2384) gains log.warn("[task-attempt] owned attempt without settlement evidence", …).
  • assert(reportOwnerWorkspaceId != null) in the new branch (agent tasks always have a parent; :2307 handles the alternative).

Change 5 — Claimed retirement

  • Claim identity (persisted in taskAttemptRetiredBy): { runId, stepId, inputHash, childTaskId, attemptId, mode, at }. Identity for idempotency is { runId, stepId, inputHash, childTaskId, attemptId } — the exact workflow checkpoint (stepId + inputHash name one record) plus the exact attempt. mode records the evidence the claim was granted on and at when; neither participates in equality, so a later claim of the same checkpoint on the same attempt is satisfied by an existing claim of either mode.
  • Monotonic. A claim is never cleared or downgraded by any code path. Its only effect is to refuse every later admission of that attempt (reawaken, reactivation, startup re-drive, queue launch) and to bind the retirement to one checkpoint; it never hides evidence — a retired child's report, if one exists, stays on disk and readable.
  • Mode is chosen by the evidence observed, not by the checkpoint's history:
    • mode: "no-report" — the prior child reads terminal-no-report (owned settlement or receipt, report artifact positively absent). CAS requires taskStatus === "interrupted".
    • mode: "retire-reported" — the prior child reads reported with a report artifact whose attemptId equals the child's current taskAttemptId. CAS requires taskStatus === "reported". Used by failed-checkpoint retries: structured-output validation failures (recordFailedAgentAttempt) and any failed record whose child was later reawakened by a user and reported. The report is not part of the run (the checkpoint already says the step failed); retiring it guarantees no reawakening runs beside the retry.
    • For a started checkpoint a reported child is always adopted (existing classifyPriorAttempt behavior); only terminal-no-report leads to disposition and a no-report claim.
  • Report artifacts gain attemptId? (SubagentReportArtifactIndexEntry, subagentReportArtifacts.ts:16–30, set at publication from the owner's attempt; legacy artifacts have none), so report evidence can be bound to the claimed attempt. retire-reported refuses artifacts without a matching attemptId.
  • TaskService.claimRetiredAttempt(taskId, attemptId, claimant: { runId; stepId; inputHash; parentWorkspaceId; mode }): Promise<Result<void, string>>, under workspaceEventLocks.withLock(taskId):
    1. Re-run the liveness checks used by inspectAttemptOutcome (latch, live registration, stream, active turn generation); any hit → Err("attempt is live").
    2. Evidence by mode, gated by lineage in both modes. Strict config read of the entry (throwOnError; failure → Err("config unreadable")). Cross-process evidence is accepted only when entry.taskAttemptUnproven !== true and a found receipt for attemptId exists (terminal settlement of the publishing owner — a bound report alone is not one: publication precedes turn settlement, and the claiming process's empty liveness maps cannot see the publisher): no-report → receipt and report artifact not_found; retire-reported → receipt and report artifact found with artifact.attemptId === attemptId. In this change a reported attempt gets a receipt only when a Stop/terminal-failure settlement follows its report (Layer 2's interrupted-while-publishing case); a receipt at ordinary post-report turn settlement is a deferred producer (below), so cross-process validation retries are usually unresolved — the conservative default. Same-process compatibility evidence, accepted regardless of the marker: the attempt is owned by this process (ownedAttemptByTaskId.get(taskId)?.attemptId === attemptId) and — for no-report — its settlement is recorded and the report artifact is not_found, or — for retire-reported — its bound report artifact is found. This compatibility path is exactly main's in-process authority and is outside the no-duplicate guarantee (a foreign process may still hold an older execution of a marked lineage); it is logged when the entry is marked. Anything else → Err("lineage unproven") / Err("no settlement evidence") / Err("report published") / Err("report evidence unreadable") / Err("report is not bound to the claimed attempt").
    3. editConfig CAS (P9): the mutator requires the mode's taskStatus, taskAttemptId === attemptId, taskAttemptUnproven equal to the value the evidence step read (the marker is only ever added, so a fresh marker means the evidence is stale → Err("claim lost"), unless the compatibility path applies), parentWorkspaceId === claimant.parentWorkspaceId, workflowTask.runId === claimant.runId && workflowTask.stepId === claimant.stepId (the child's own association, config/index.ts:3549), and either taskAttemptRetiredBy == null (set it) or an existing claim with equal identity (any mode/at; idempotent under any lease holder — the mutator leaves it untouched); anything else → Err("claim lost").
    4. no-report only, defense in depth: re-read the report artifact after the CAS. not_foundOk. found or unreadablethe claim stays and the call returns Err("report-after-settlement") / Err("report evidence unreadable"); the runner treats both as unresolved (below). Within the guarantee boundary this branch is unreachable — a receipt or owned settlement is written only after the owner's last publication (P1), in-process publication is serialized by the same event lock, and any other process's admission of this attempt is excluded by the CAS (P9) — so it is logged at error level as an invariant violation, never "handled" by clearing the claim.
  • Lifecycle. The old child is permanently retired for admissions (refused with a stable message naming run/step); it remains inspectable and removable (task_remove). Ordering on the started path is: read → recordStartedAttemptFailed (inside classifyPriorAttempt) → restart()reserveAgentTasksclaimcreateAgentTasks (checkpoint started naming the new child → config commit). Crash after disposition, before the claim → the checkpoint is failed naming the old child; the next retry takes the failed-checkpoint path, reads the child, claims by evidence and replaces once. Crash after the claim, before the checkpoint → same, and the identical claim is idempotent. Crash after the checkpoint → the checkpoint names the replacement; existing semantics apply (queued replacement launched by startup → owned → live; config entry never committed → indeterminate, deferred tombstone). A different checkpoint (other run, other step, other inputHash) can never consume the claim. Lease loss after an awaited claim: the runner's existing leaseGuard.throwIfLost() before reservation stops the runner; the claim persists for the next holder of the same checkpoint, which reuses it. Duplicate replacement reservations remain prevented by the existing lease and fenced checkpoint writes (WorkflowRunStore attempt/run-state fences) plus checkpoint-before-commit (P10).
  • One chokepoint: replacement reservation. WorkflowRunner.reserveAgentTasks (:2793) is the only place children are created. It gains priorChild?: { taskId: string } per step, filled by all three callers that replace — runOrResumeAgentStep (:2412, threaded through restart()), pipeline startAgentStep (:1375), and the failed-step re-run (:1242, which today reserves without consulting the old child). Before createAgentTasks, for each priorChild: outcome = readSettledAgentResult(taskId); then
    • outcome.identity.kind === "legacy" — the strict config read succeeded and found an existing entry with no taskAttemptId (positive proof of a pre-upgrade child) — and the outcome is reported or indeterminate with code: "legacy-no-attempt-id" (not report-unreadable, receipt-unreadable, config-unreadable or any other evidence failure): on the failed-checkpoint path proceed without a claim (today's behavior, logged as the explicitly accepted legacy exemption); on the started path unresolved (terminal-no-report always carries an id after Change 4). identity.kind === "unknown" (entry missing, config unreadable, evidence unreadable) → unresolved on both paths; removed children stay deferred to the tombstone follow-up and are not exempt.
    • terminal-no-reportclaimRetiredAttempt(…, mode: "no-report"); reported with a bound artifact → claimRetiredAttempt(…, mode: "retire-reported"); live / cleanup-pending / indeterminate / reported without a bound artifact → WorkflowPriorAttemptUnresolvedError(stepId, taskId, reasonCode, reason) (run re-interrupted, checkpoint intact, no reservation).
    • Err from the claim (claim lost, attempt is live, report-after-settlement, report evidence unreadable, …) → WorkflowPriorAttemptUnresolvedError with that reason; never a retry loop inside the runner.
    • After the awaited claim: leaseGuard.throwIfLost() and the batch/run abort signals are re-checked before any reservation.
    • Capability is mandatory. The adapter interface gains claimRetiredAttempt as a required member; the real adapter delegates to the service. Test stub adapters are updated to return attemptId and to record claim calls (test-only change) — no runner branch keeps "today's behavior" when an id is present.
      classifyPriorAttempt's terminal-no-report case therefore only records the step failed and returns replace; the claim happens at reservation. Covers Stop-drain (disposeStartedAttempt records failed without a claim) → restart → user reawaken → retry: the reawakened child is live, the retry stays unresolved until it is stopped (then no-report) or reports (then retire-reported).
  • WorkflowTaskServiceAdapter: required claimRetiredAttempt(taskId, attemptId, claimant) delegating to the service; readSettledAgentResult passes attemptId through on every outcome.
  • Admission refusal: markInterruptedTaskRunning, reactivateChildAgentTask and the startup re-drive rotation refuse inside their CAS when taskAttemptRetiredBy != null, with the stable message "This sub-agent's attempt was retired by workflow run <runId> (step <stepId>); start a new task instead." surfaced through the existing tool error paths (task_send_message reawaken, task_await).

Files

src/node/config/index.ts; src/node/services/taskService.ts; new src/node/services/subagentAttemptSettlements.ts; src/node/services/subagentReportArtifacts.ts (attemptId? at publication); src/common/types/tasks.ts; src/node/services/workflows/WorkflowTaskServiceAdapter.ts; src/node/services/workflows/WorkflowRunner.ts. Tests: taskService.test.ts, new subagentAttemptSettlements.test.ts, WorkflowRunner.attemptDisposition.test.ts, WorkflowTaskServiceAdapter.test.ts, new cross-process fixture src/node/services/__fixtures__/attemptRecoveryFixture.ts + taskService.attemptRecovery.crossProcess.test.ts.


Tests first (red → green)

Unit (taskService.test.ts, describe "attempt outcome and settlement"; harness createTaskServiceHarness(config) :220–260)

  1. Identity. Reservation commit persists taskAttemptId and the owned attempt carries it; a same-process launch keeps it; a launch of a plan not reserved by this process rotates it; reawaken and reactivation persist fresh ids; a reactivation whose CAS fails restores speculative memory and publishes nothing; a reactivation whose CAS committed keeps the new id in config and memory whether createWorkspaceTurn returns Err or throws, reads indeterminate (owned, unsettled), and a later Stop settles it with a receipt; a pre-upgrade entry without id gets one at launch with taskAttemptUnproven.
    1d. Lineage. Reservation → receiptEligible: true, no marker; drain launch of a foreign queued entry without marker → eligible; stale-starting revert sets the marker and the subsequent drain launch is owned but not eligible; startup re-drive rotation sets the marker; reawaken with a receipt for the current id → eligible, marker omitted; reawaken with the same process's eligible settlement → eligible; reawaken of a marked entry, of an id without receipt, or with an unreadable receipt → owned but not eligible, marker persisted; reawaken while the predecessor's latch is retained waits (bounded, outside the event lock) and then decides; a pre-upgrade entry → marked. Marker race: a fixture hook sets taskAttemptUnproven on the entry between an admission's proof snapshot and its CAS (reawaken, reactivation, and drain launch) → the committed attempt is receiptEligible: false, the marker is persisted, and no receipt is written at settlement.
    1e. Settlement closes admission (the interval, not its aftermath). Gate mapping: in G1 (no producers) every "receipt" below reads as "the settlement entry upgrades to settled / the stop record releases"; the receipt-specific variants (blocked durable write, receipt file contents) are G2 and live in test 6/6c. (i) Idle stop with the settlement upgrade held (fixture hook between closure and settleOwnedTaskAttempt; in G2 the hook is inside persistOwnedAttemptSettlement): a same-attempt continuation parked before the fence is released while the upgrade is held → refused (closing entry), zero provider dispatches for X; the upgrade completes → settled (G2: receipt X), config id unchanged; a later user reawaken rotates to Y and dispatches. (i-b) Unowned supersession: startup re-drive Y parked in preflight (workspaceService.ts:11982); a second rotation publishes Z (currentAttemptIdByTaskIdZ); release Y → refused at :4791, zero admissions under Y; Z dispatches. (i-c) Disposed token is terminal: cancel a pending token (onDisposed("canceled-before-admission")) while its attempt stays open, then drive its dequeue/prepare → refused; the attempt's next fenced send admits normally. (i-d) Legitimate reawaken admits: owned X settled (settled phase) → reawaken publishes Y → the fence's common predicate passes for Y, ownership of Y is installed in the same block, the send is admitted (onAdmitted), and X's settlement entry is gone while a closure recorded for Y between CAS and block (fixture) makes the block refuse and is not erased. (ii) Opposite order: the continuation passes the fence first (one AdmittedSend pending, turn not yet visible); a Stop begins → Phase A captures the pending admission, no receipt until that turn becomes visible, is stopped by the owed second stopStream, and settles → exactly one receipt, after the late turn's settlement. (iii) failAgentTaskTerminally begins (closure recorded) → a parked continuation is refused before the receipt exists. (iv) Launch failure racing a fence: fence first → sendAdmitted → no receipt; markTaskLaunchFailed first → closure → the fence refuses → receipt. (v) Pending send with a captured turn: T1 visible and captured by a Stop; an admitted send is still pending (turn not yet visible); T1 settles and cleanup completes → no receipt until the pending send is refused or its turn T2 becomes visible, is stopped by the owed second stopStream, and settles; the receipt then appears once. (vi) Blocked receipt write, observable: with the idle receipt write blocked outside the lock, readAttemptOutcome returns cleanup-pending (not a lock wait) and a prepared continuation returns Err("attempt settled") immediately. (vii) Unowned startup re-drive passes the fence: recoverInterruptedTasks re-drive dispatches through the new-attempt branch with a fresh marked id and no ownership; the same task with taskAttemptRetiredBy set is refused and skipped. (viii) Queued input after closure: a message queued into a live child before Stop is refused by the fence after closure and never starts a turn. (ix) Pending send through a Stop, at the real gate: a reawaken (owned) and a startup re-drive (unowned) publish Y and enter sendMessage; park inside its awaits (sessionInvisiblePreflight.awaitEarlierDecisions, workspaceService.ts:11982); Stop begins → closure for Y / Phase A captures the pending obligation → no receipt can complete while it is pending; release → the token's admissionStale refuses at agentSession.ts:4791, onDisposed("refused"), zero coordinator.prepare admissions → the record releases → receipt appears after the refusal, never before; Y stays published, config unchanged by the fence. (x) Admitted obligations survive supersession and cancellation; refused ones never admit: (a) a send under Y is admitted (onAdmitted(turnId)) and then Y is superseded by ZY's obligation stays admitted until its turn settles, is captured only by records bound to Y, never enters Z's record; (b) an admitted (PREPARING) send is canceled → onDisposed ignored, obligation retained, no receipt until the turn settles; (c) an enqueued send under Y whose dequeue happens after closure → refused at :10128, onDisposed("refused"), never admitted, zero dispatches under Y. (xi) Lifecycle table at the real seams: resumeStream returning Ok({ started: false }) leaves no pending obligation; dedupe leaves the new token no-work and the existing item's token intact; a dequeued item admitted at :10128 whose :10188 re-entry fails early is admitted (not "never admitted") and its turn's settlement discharges it; a send that returns Ok while another turn is live has exactly one enqueued obligation whose later dequeue fires onAdmitted once (the :10188 adoption is idempotent); double cancellation/completion never double-discharges.
    1b. Admission audit. Every startup re-drive path rotates the id before its send and is refused when claimed; same-attempt continuations (in-owner recovery prompts, guidance into a live child, owned kickoffs) leave the id unchanged; each classified site has a test that observes the id before/after.
    1c. Claim vs. admission, both orders. Reawaken CAS committed first → claim Err("claim lost"); claim committed first → reawaken/reactivation/startup rotation refused with the stable message; asserted through the real editConfig, not mocks.
  2. Receipts only from owners. Each settlement site writes exactly one receipt for the captured id (idle stop, stop-record release — assert durable before the latch releases run —, launch failed, reservation canceled/failed, terminal failure). Negative: stopping an unowned task (second harness instance) writes no receipt (:33347–33370 extended); startup re-driven task that fails terminally writes no receipt; an owned but receiptEligible: false attempt (reawakened from a marked entry, or launched from a reverted starting entry) settles in memory (same-process terminal-no-report) and writes no receipt.
  3. Delayed settlement after re-admission (P2). Attempt 1 enters failAgentTaskTerminally with its captured attempt; before its write, reawaken as attempt 2; the write must produce a receipt named attempt 1 only, settleOwnedTaskAttempt refuses in-memory, classification of attempt 2 stays live/indeterminate.
  4. Classification. Prior-process interrupted + receipt for current id → terminal-no-report with attemptId; receipt for an older id → indeterminate; no id → indeterminate (legacy); receipt unreadable → indeterminate; report present → reported; report unreadable → indeterminate; strict config read throws → indeterminate; id/status changed between reads → indeterminate; entry archived → terminal-no-report.
  5. Claim. Succeeds once; idempotent for the same { runId, stepId, inputHash, childTaskId, attemptId } under a new lease; refused for another run, step or inputHash; refused when a reawaken changed the id between read and claim; refused when the task became live; refused (lineage unproven) for a marked entry with a receipt-less predecessor or a bound report, in both modes, from a non-owning process; accepted for the same marked entry from the owning process (compatibility) with the log line; refused (claim lost) when the marker appears between evidence read and CAS; config unreadableErr; reawaken, reactivation and startup rotation refuse after a claim with the stable message; report found after CAS → Err("report-after-settlement") with the claim retained; an existing no-report claim satisfies a later retire-reported claim of the same identity (and vice versa) without a second write; a claim is never cleared by any code path (grep-level test: no writer sets taskAttemptRetiredBy = undefined).
  6. Stop-path accounting (P7). Receipt write blocked → latch stays held (cleanup-pending, cleanup owed) past the stop aggregate deadline (the Stop caller returns, the latch is retained) → write completes late → latch releases exactly once and the attempt reads terminal-no-report; a second recheck while pending schedules no second write; a rejected write pays back the cleanup, records in-memory settlement with durable: false, and the attempt reads indeterminate from a fresh process; service disposal with a pending write neither drops nor double-counts it.
    6b. Publication boundaries. For each producer in the boundary table, a test holds the attempt at the boundary (report published during Phase B; stream end carrying a report; idle-stop branch with a preparing turn; launch failure after admission; reactivation Err vs. throw) and asserts reported/no receipt where publication was still possible.
  7. Downgrade/field loss. Entry with taskAttemptId stripped (simulated downgrade write) → indeterminate; next admission assigns a fresh id; the old receipt never matches.

Runner (WorkflowRunner.attemptDisposition.test.ts, stub adapter)

  1. terminal-no-report with attemptIdrecordStartedAttemptFailed first, then exactly one claimRetiredAttempt({ runId, stepId, inputHash, mode: "no-report" }) inside reserveAgentTasks before createAgentTasks; claim ErrWorkflowPriorAttemptUnresolvedError, run interrupted, checkpoint (failed, old child) intact, no reservation; failed-checkpoint retry with a reported prior child and bound artifact → retire-reported claim then one reservation; reported without bound artifact → unresolved; failed-checkpoint retry with identity.kind === "legacy" → proceeds unclaimed (logged); identity.kind === "unknown" (missing entry / unreadable) → unresolved on both paths; started path with legacy → unresolved. Existing stub adapters updated to supply attemptId and record claims.

Cross-process (new, Bun.spawn of a fixture script over a temp XUM_ROOT)

  1. Decisive regression. Process A (fixture): real TaskService/WorkflowTaskServiceAdapter/WorkflowRunner/WorkflowRunStore; two-step workflow; step 1's child reports; step 2's child is admitted and then stopped through TaskService while the runner's wait is parked at a fixture barrier; A prints a marker once all hold: run.json step 2 started, child interrupted with taskAttemptId, receipt file present, run lease still held. The test then SIGKILLs A (no interruptRun, so the step stays started), verifies the pre-restart state on disk, constructs the services in the test process over the same root, runs recoverInterruptedTasks(), resumes → step 1 reused (no new task), exactly one replacement reserved (fresh taskAttemptId), taskAttemptRetiredBy set on the old child, task events [old, failed] → [new, started], old id never awaited.
  2. Live foreign publisher. Process A keeps the child live (stream parked) and stays alive; the test process constructs its services without running startup re-drive on that child (so the config id stays A's X, unmarked), reads indeterminate, resume → unresolved, no reservation; after A's stop settles (receipt X) → the test process reads terminal-no-report and replaces exactly once. (With startup re-drive the id rotates to a marked Y and the lineage is fail-closed forever — test 19(a).)
  3. Crash before receipt. A killed after interrupted is persisted but before the receipt (fixture barrier between the config write and the receipt write) → indeterminate, unresolved, checkpoint intact, second attempt to resume is identical (idempotent).
  4. Second restart (replacement stopped in the test process → its receipt → a third instance replaces exactly once more) and reawaken between classification and reservation (test-process reawaken racing the resume → claim lost or admission refused, never both a reawakened child and a replacement).
  5. Startup redispatch invalidates a receipt. Fixture leaves a receipt for attempt X and a config status the startup path re-drives (running/awaiting_report, produced by the fixture after the receipt); the test process's recoverInterruptedTasks() rotates the id before sending; classification then reads indeterminate (receipt X no longer matches), never terminal-no-report.
  6. Crash after claim, before the checkpoint names the replacement: kill after taskAttemptRetiredBy is durable (the step is already failed, disposition precedes the claim) and before the started checkpoint; the next retry takes the failed-checkpoint path, re-claims idempotently and replaces exactly once. Crash after replacement reservation, before launch: kill after the started checkpoint names the replacement; the next process launches it (owned, live) and no further replacement is reserved. Crash after disposition, before the claim: kill after recordStartedAttemptFailed is durable; the next retry takes the failed-checkpoint path, claims, replaces once. Crash between checkpoint and config commit (fixture barrier inside onTaskReserved): the next resume reads indeterminate("no task record …"), unresolved, no duplicate reservation.
  7. Foreign publisher at the publication boundary. Process A is held inside handleStreamEnd right before publishAgentTaskReport; the test process reads indeterminate/live and a claim fails; A then publishes → reported is adopted, no receipt is ever written for that attempt.
  8. Receipt write completes after the stop deadline (cross-process variant of test 6): A's receipt write is held past the Stop caller's deadline; A reads cleanup-pending (its latch), the test process reads indeterminate (no receipt yet, no access to A's latch); both refuse replacement; A's write completes → the test process reads terminal-no-report; A's latch released once.
  9. Failed checkpoint after reawaken. A Stop-drain records step failed (no claim); restart; the test process reawakens the old child (live); a checkpoint retry must not reserve a replacement (unresolved with the "prior child is active" reason); stopping the reawakened child (receipt) → retry claims (no-report) and replaces exactly once; variant where the reawakened child reports → retry claims retire-reported, replaces once, the report artifact remains on disk.
  10. Terminal failure with a live stream. Fixture holds A's child stream open after failAgentTaskTerminally persisted interrupted; the test process reads indeterminate (no receipt yet); A's stream end publishes agent_reportreported, never a receipt; variant without a report → receipt after turn settlement → terminal-no-report.
  11. Foreign process retains publishing authority → no replacement. (a) A admits a child (starting/running) and stays alive; the test process's startup reverts/re-drives it unowned; after A's child settles without a report, the test process reads indeterminate (no receipt of its own; A's receipt, if any, is for A's attempt and A's stream may still publish) and a resume stays unresolved; and it stays unresolved after A itself settles X (receipt X) — the re-drive rotated the config id to the marked Y, so receipt X never matches; recovery of that lineage exists only inside the process that owns it (compatibility path), consistent with (c) and test 13. (b) A holds a queued reservation (id X) and stays alive; both processes race the queued → starting CAS; exactly one commits and sends; the loser never sends and never writes a receipt; the winner's receipt (X or rotated Y) is the only one. (c) Laundering attempts. A admits a child (starting) and stays alive with its stream parked; the test process's startup reverts it to queued (marker set) and its drain launches it (owned, not eligible); the test process stops it → no receipt; the test process then reawakens it → still not eligible → stop → no receipt; a third instance reads indeterminate throughout. Same with A's child running: startup re-drive (marker) → recovery limit → reawaken → stop → no receipt. After A itself settles X (receipt X) nothing changes for the marked lineage: it stays fail-closed. Reported successor: the reawakened marked child reports (artifact bound to its id) and B exits; a third instance's failed-checkpoint retry reads reported but the claim returns Err("lineage unproven") → unresolved, no replacement; the same retry in B (owner) succeeds via the compatibility path and is logged as outside the guarantee.
  12. Validation retry through the real adapter. Same process: prior child reported an invalid output (artifact bound to its attemptId); the retry claims in retire-reported mode via the compatibility path and replaces once. Cross-process boundary: A publishes the bound report and reported, then parks before turn settlement; B's validation retry reads reported and the claim returns Err("no settlement evidence") → unresolved, no reservation; A resumes and settles normally → still no receipt (deferred producer) → B stays unresolved; variant where A's child is Stopped while its stream end publishes the report (receipt + bound report) → B retires and replaces once; a reawaken of the reported child racing the retry → claim lost → unresolved; a legacy reported child without taskAttemptId → retry proceeds unfenced (today's behavior, logged).
  13. Report discovered during a no-report claim (invariant-violation drill, artifact injected by the fixture). Receipt present, report artifact appears between the read and the post-CAS re-read → Err("report-after-settlement"), claim retained, run unresolved, no replacement, error-level log; reawaken of the child refused; the report artifact is untouched.
  14. Lease turnover during a claim. Runner A claims and is parked before the post-CAS re-read (fixture barrier); its lease goes stale; runner B (test process) reuses the identical claim and reserves the replacement; A is released and completes its re-read → A's outcome (Ok or Err) never touches taskAttemptRetiredBy; exactly one replacement exists.

Keep green unchanged: :33278–33316, :33347–33370, :33435–33462, full taskService.test.ts, WorkflowRunner.attemptDisposition.test.ts, WorkflowRunner.test.ts, WorkflowRunStore.test.ts, WorkflowTaskServiceAdapter.test.ts, WorkflowService.test.ts, config tests over the explicit field lists.

Acceptance

  • A workflow resumed or retried after a restart replaces a retired attempt at most once, and does so on every successful replay to a checkpoint whose child's proven owner settled it with a receipt and whose report is positively absent (the checkpoint-without-config-entry crash gap stays unresolved); reused reports, completed steps and live children are untouched; the retired child cannot be reawakened or reactivated afterwards. Replacement is never admitted across processes for a marked (unproven) lineage in either claim mode.
  • Attempts without a matching receipt remain indeterminate with an explicit reason; the run stays interrupted with its checkpoint intact; resuming again is idempotent.
  • No admission path can start work for a claimed attempt; no receipt is ever written for an attempt the writing process does not own or whose lineage is unproven; a claim, once written, is never cleared.
  • Every evidence read/write failure fails closed; same-process behavior is unchanged apart from the receipt write preceding in-memory settlement.
  • One new artifact directory, three optional config fields (taskAttemptId, taskAttemptUnproven, taskAttemptRetiredBy), one runner branch guarded by the claim.

Quality gates

  1. Write the tests → bun test src/node/services/taskService.test.ts --test-name-pattern "attempt outcome and settlement", bun test src/node/services/subagentAttemptSettlements.test.ts, bun test src/node/services/workflows/WorkflowRunner.attemptDisposition.test.ts, bun test src/node/services/taskService.attemptRecovery.crossProcess.test.ts → confirm red.
  2. G1 — admission lifecycle prerequisite (Changes 1, 2, 3a), receipt-independent. Implement identity, lineage, the TurnAdmissionToken wiring and closure/obligation accounting with no receipt producer enabled. G1 asserts only: real admission/refusal at the three gates, queue handoff and dequeue re-check, cancellation dispositions, exact turn correlation and settlement, eventual discharge with no orphaned obligation (every obligation ends discharged or is retained by a record that eventually releases), closing/settled phases and cleanup-pending, lineage flags (receiptEligible) and marker persistence. Tests: 1, 1b, 1d, 1e (G1 reading), 12/13's identity assertions, plus admission refusal against a seeded taskAttemptRetiredBy field (no claim implementation yet), plus the sibling agentSession/workspaceService/messageQueue suites; make static-check. Test 1c (real claim CAS vs. admission) and every test that exercises claimRetiredAttempt belong to G2. Observable behavior unchanged except: same-attempt continuations after a Stop/closure are refused (logged), and cleanup-pending is reported while a closure is pending. A run in which an obligation can never be dispositioned is a failing G1, not a passing fail-closed one.
  3. G2 — receipts, claims, replacement (Changes 3b, 4, 5). Implement only after G1 is green; tests 1c, 2–11, 14–22 and 1e's G2 variants green; make typecheck && make lintmake static-check.
  4. Sibling suites: full taskService.test.ts, WorkflowRunner.test.ts, WorkflowRunStore.test.ts, WorkflowTaskServiceAdapter.test.ts, WorkflowService.test.ts, tools/workflow_resume.test.ts, tools/task_send_message.test.ts, src/node/config/*.test.ts.
  5. Dogfood (below) with screenshots and video; verify persisted run.json, config.json and receipt files, not only the UI.
  6. Independent readiness recommendation from a clean-context reviewer given the diff, test output and dogfood evidence; then the readiness decision.

Conventions: read the pull-requests skill before any commit; nothing is pushed or opened without an explicit request. Deliver as a two-PR stack matching the gate: PR-A = Changes 1, 2, 3a (identity, lineage, receipt module without producers, admission lifecycle contract; no observable classification change beyond the refusals noted in G1), PR-B = Changes 3b, 4, 5 (receipt producers, classifier, claims, runner). PR-B is opened only after PR-A's G1 evidence is recorded.

Dogfooding (isolated sandbox, restart scenarios)

Setup:

  1. KEEP_SANDBOX=1 make dev-server-sandbox DEV_SERVER_SANDBOX_ARGS="--clean-providers --clean-projects" as a monitored background bash (dev-server-sandbox skill); record XUM_ROOT, BACKEND_PORT, VITE_PORT.
  2. Seed the sandbox providers.jsonc with a loopback OpenAI-compatible fixture (pattern: tests/ipc/providers/openaiCompatible.test.ts), scripted: the parent's first turn calls workflow_run with an inline two-step agent workflow; step 1's child calls agent_report; step 2's first child streams text and never calls agent_report; step 2's replacement child calls agent_report. Enable llmDebugLogs for provenance.
  3. Open the UI with an owned agent-browser session (--session recovery-dogfood; never close --all); create a project/workspace; send the parent prompt.

Scenario 1 — child stopped, then restart (the guaranteed case):

  1. While step 2's child streams, Stop the child from the UI; screenshot child interrupted, run interrupted; confirm sessions/<parent>/subagent-attempt-settlements/<child>/<attemptId>.json exists and matches config.json's taskAttemptId. Stop the sandbox server (task_stop on the monitored bash); restart with the same root (XUM_ROOT=<sandbox> BACKEND_PORT=… VITE_PORT=… make dev-server, monitored).
  2. Reopen the workspace, workflow_resume (mode resume) from the parent chat. Expected: exactly one replacement child, run completed; run.json task events [child_old, failed] → [child_new, started] → [child_new, completed], step 1 reused; config.json shows the old child interrupted with taskAttemptRetiredBy naming the run/step, the new child reported. Then task_send_message to the old child → refused with the retirement message. Screenshots of the completed card and the refusal.

Scenario 2 — restart while the child is running (partial: needs a checkpoint retry):

  1. New run; while step 2's child streams, stop the server; restart; reopen the workspace (crash recovery resumes the run). Expected: startup re-drives the child without ownership and marks it taskAttemptUnproven; it hits the recovery limit → interrupted, no receipt → the resumed run's reattach wait rejects, handleAgentWaitFailure reads indeterminate → run failed with the child's (masked) error; retry_from_checkpointindeterminate → unresolved. Then the same-process recovery that works on main: task_send_message reawakens the child (owned, marker kept), Stop it, retry_from_checkpoint → replaced once in this process (no receipt file appears). Restart once more and repeat with a fresh run: after the restart the marked lineage stays indeterminate. Document this as the known partial case (ownership handoff deferred); screenshot the diagnostic and the config marker.

Scenario 3 — negative (no receipt):

  1. New run; Stop the child; before restarting, delete the receipt file. Restart, resume. Expected: run stays interrupted with the indeterminate diagnostic naming the missing receipt; no replacement; resuming again is idempotent. Screenshot.

Evidence: PNGs per numbered step, a video (agent-browser record start/stop) of scenario 1 from restart to completion, quoted run.json task events, receipt JSON, and config.json fields; attach_file all of it. Tear down only the owned sandbox and browser session.


Deferred follow-ups (tracked separately; not part of this change)

  • Ownership handoff for startup re-driven tasks so their terminal settlement produces receipts (recoverInterruptedTasks :3301–3423): requires establishing that the previous execution cannot publish before beginOwnedTaskAttempt — the same proof this change declines to infer.
  • Operator-authorized replacement for attempts without a receipt (workflow_resume({ replace_unresolved_attempts: true }) + UI action), gated to interrupted + report positively absent, with the assertion made explicit in tool output.
  • Structural checkpoint-retry eligibility and QuickJS error normalization (workflowRetryEligibility.ts:21–24, quickjsRuntime.ts:1511–1517): needs typed failure provenance; eligibility and normalization must change together.
  • In-process owned attempts that become interrupted without a settlement (interruptTaskRecoveryForInactiveWorkflowOwner :2590–2602, plan-schema rejection :13639–13651); the new log.warn makes occurrences visible.
  • Removed children (entry == null) via removed-agent-tasks/<taskId>.json after verifying the tombstone certifies completed removal; also covers a started checkpoint whose child's config entry was never committed (crash between checkpoint and commit).
  • Receipt at ordinary post-report turn settlement (recordWorkspaceTurnSettled for an owned, eligible, reported attempt) so cross-process validation retries can retire a reported attempt: needs a focused audit of the post-report turn lifecycle (queued input dispatched after agent_report, continuation kickoffs of a reported best-of candidate) to establish that turn settlement is the last publication/admission boundary; until then cross-process retire-reported is unresolved by default.
  • Receipts for queued children interrupted by the inactive-owner prepass (interruptTaskRecoveryForInactiveWorkflowOwner, crash between a run's terminal status write and its reservation cancel): provable (queued without marker ⇒ never admitted; the interrupt CAS excludes any later admission of that id) but a distinct producer with its own proof row; not needed for the historical failures.
  • Failure-tolerant parallel for deep-research votes.

Accepted trade-offs

  • Partial guarantee by design: legacy, crash-before-receipt and unproven lineages stay fail-closed across processes. Children found starting/running/awaiting_report at a restart (and everything later reawakened from them) are marked unproven and never produce receipts; the same-process "reawaken, then Stop, then resume" recovery that works on main today keeps working for them, but a further restart leaves them indeterminate. Children found queued without the marker are launched under an exclusive CAS and are fully covered.
  • In-process ownership semantics are unchanged from main: a process that admits an attempt (proven or not) treats its own recorded settlement as authoritative for its own runners. Two concurrent backends on one root remain outside the guarantee for in-process authority, exactly as today; the new cross-process authority (receipts, claims) never widens that exposure.
  • Legacy prior children (no taskAttemptId) in failed-checkpoint retries proceed without a claim, exactly as today. This is the only unclaimed replacement path.
  • A failed checkpoint whose prior child was reawakened by a user and reported is retried by retiring that reported attempt (retire-reported), not by adopting the late report; the report stays on disk.
  • Cross-process validation retries (failed checkpoint, prior child reported, owner process gone) stay unresolved unless a receipt exists for the reported attempt; same-process validation retries work as on main.
  • A same-attempt continuation prepared before a Stop/terminal failure is dropped at the admission fence once the attempt settles (logged); on main it could dispatch after the settlement.
  • A retired (claimed) child can no longer be reawakened; its workspace stays inspectable.
  • On the stop path, in-memory settlement now waits for the receipt write to settle; a write that stalls keeps the latch retained (cleanup-pending) rather than being abandoned, and a confirmed failure degrades only cross-restart recovery.
  • Startup re-drive rotates the attempt id (one extra config write per re-driven task at startup, bounded by the number of active tasks — the same set recoverInterruptedTasks already writes for).
  • Receipt files accumulate per attempt that ended without a report (small, immutable); retention is bounded by the same set as the in-memory receipts today.

Deployment decision (open)

Single PR or a two-PR stack (identity+receipts, then claim+runner). Children failing with recovery-limit errors remain checkpoint-retryable through the existing message match; other child failure messages still require a fresh workflow_run until the deferred eligibility follow-up lands.


Generated with xum • Model: coder:openai/gpt-6-astra • Thinking: xhigh • Cost: $533.73

Merge pinned main 60d4039 without rewriting published G1 history.
Keep both independent additions to agentMessaging constants; the other
files merge automatically. Integration validation is recorded separately
from the daed443 remote UAT snapshot.

---

_Generated with [`xum`](https://github.com/coder/xum) • Model: `coder:openai/gpt-6-astra` • Thinking: `xhigh` • Cost: `$533.73`_

<!-- mux-attribution: model=coder:openai/gpt-6-astra thinking=xhigh costs=533.73 -->
@ThomasK33
ThomasK33 marked this pull request as ready for review September 20, 2026 21:06
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 20, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-20T23:24:58.760058Z d40de5c New commits
🔒 Security Review Completed 2026-09-20T23:18:48.247536Z d40de5c New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0333deb1c8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/taskService.ts Outdated
Comment thread src/node/services/taskService.ts Outdated
Comment thread src/node/services/agentSession.ts Outdated
Comment thread src/node/services/taskService.ts Outdated
Comment thread src/node/services/taskService.ts
Comment thread src/node/services/taskService.ts
Comment thread src/node/services/taskService.ts Outdated
A lifecycle observer may synchronously admit or settle a successor.
Capture the transitioning generation and update its observation before any
callback, so the predecessor cannot settle the successor or overwrite its
live/idle state. Notify supersession before publishing nested transitions.

Two AgentSession regressions fail on the reviewed head and pass after the
fix. The full session/coordinator suites and make static-check also pass.
Addresses Codex discussion_r4058106801 on PR #4308.

---

_Generated with [`xum`](https://github.com/coder/xum) • Model: `coder:openai/gpt-6-astra` • Thinking: `xhigh` • Cost: `$604.72`_

<!-- mux-attribution: model=coder:openai/gpt-6-astra thinking=xhigh costs=604.72 -->
…direct launches

Corrections to the G1 attempt-admission layer in TaskService, each reproduced
red first (taskService.attemptAdmission.test.ts, 8 new tests):

- markInterruptedTaskRunning: ownership follows the committed CAS (as the
  reactivation path already does). A reawaken losing its CAS to a concurrent
  one used to roll the in-memory mirror back to the predecessor id while config
  and ownership named the winner, so the next manual send was admitted against
  the retired id; a send racing the CAS was bound to a not-yet-persisted id.
- currentTaskAttemptId: the persisted row is authoritative; the mirror is only
  a fallback for a row that lost its id. Tokens of an attempt another writer
  rotated are revoked at their next gate instead of riding the stale mirror.
- Direct (unqueued) create: the attempt id stamped by the entry write is now
  owned by this process (first admission by construction, receipt-eligible),
  and every rollback settles it as launch-failed. Previously the task tool's
  primary spawn path left its children unowned, so a Stop closed but never
  settled them (indeterminate) and their lineage could never be proven.
- rotateAttemptForStartupRedrive: CAS on the recovery snapshot's id and status
  so a row another writer admitted or stopped meanwhile is skipped rather than
  overwritten and re-driven.
- admitTaskWorkspaceTurn: a workflow claim refuses before the id check, so a
  retired task with a missing or malformed id fails closed.
- evaluateAttemptLineage: a receipt at the parent's path must name that parent.

---

_Generated with `xum` • Model: `coder:anthropic/claude-fable-5-1` • Thinking: `xhigh` • Cost: `$7.03`_

<!-- mux-attribution: model=coder:anthropic/claude-fable-5-1 thinking=xhigh costs=7.03 -->
… owner

A Stop cascade whose Phase A runs after a reawaken's identity CAS but before
beginOwnedTaskAttempt captured either the superseded predecessor (write still
in flight) or the fresh id unowned (commit visible). Its release then closed
an id nobody owned or settled a predecessor nobody held, leaving the new owner
permanently indeterminate with its id open or closed-but-unsettled.

beginOwnedTaskAttempt now rebinds a live stop record for the task to the
attempt being installed, so the cascade's existing Phase C settles the attempt
that is current (terminal without report, closed to sends, lineage proven for
the next reawaken). Nothing can run under that attempt meanwhile: the latch
refuses every admission until release. Current-id closures recorded in the
window are preserved as before.

currentTaskAttemptId no longer falls back to the id this process remembers: a
row that is missing, lost its id, or loads as the default view yields no
attempt, so every token reads stale and every fence refuses instead of
reviving a stale memory. currentAttemptIdByTaskId keeps its one read, the
fence's fail-closed path on an unreadable registry.

Deterministic witnesses (Phase B gated on stopStream) cover the Stop landing
before and after the commit becomes visible, and the deleted/unreadable-row
cases; each is red without its change.

---

_Generated with `xum` • Model: `coder:anthropic/claude-fable-5-1` • Thinking: `xhigh` • Cost: `$7.03`_

<!-- mux-attribution: model=coder:anthropic/claude-fable-5-1 thinking=xhigh costs=7.03 -->
…pture

Replaces the stop-record rebind from the previous commit. Every cascade's
Phase A (beginWorkspaceStop) runs under TaskService's global mutex, so the
reawaken (markInterruptedTaskRunning) and the reactivation
(reactivateInactiveAgentTask) now run {latch recheck, identity CAS,
post-commit row check, publish, beginOwnedTaskAttempt} inside that same
mutex: no Stop can observe the fresh id, or the superseded predecessor,
between the commit and its owner. Lineage evaluation (receipt read, bounded
wait), metadata emission and the caller's send stay outside. A Stop that
completes while a reawaken is still evaluating overtakes it (stop-epoch
fence: refused, nothing rotated; a recovery started afterwards proceeds); a
reactivation into a task whose cascade is already latched is refused before
it publishes. A row rotated by a writer outside the mutex after the commit
is never republished or owned.

Lock audit: mutex holders that send (create's launch, WorkspaceTurnManager's
continuation) must never reach the rescue. WorkspaceTurnManager sends carry
their correlation, which WorkspaceService already exempts; the direct and
reserved launch sends now bind their obligation and pass the token as the
send's staleness probe, so WorkspaceService treats them as guarded sends and
skips the user-resume rescue by its existing rule. Lock order is mutex →
desktop gate → config queue, the one create/createWorkspaceTurn establish;
no event or tree lock is taken inside.

Witnesses pause inside the critical section (during the CAS write and after
the id is durable), request a real Stop and prove it cannot capture until
the owner installs, then complete it: exact new-id settlement, latch and
record gone, closed-id refusal, next reawaken proven. Also: a gap send binds
to the committed id and is drained before release; a completed Stop overtakes
an in-flight reawaken; a non-mutex successor is never overwritten; a throwing
CAS releases the mutex. Two baseline tests whose fake host re-entered the
rescue under the task-creation lock now model the serialized ordering, and
launch-option assertions match the guarded send.

---

_Generated with `xum` • Model: `coder:anthropic/claude-fable-5-1` • Thinking: `xhigh` • Cost: `$7.03`_

<!-- mux-attribution: model=coder:anthropic/claude-fable-5-1 thinking=xhigh costs=7.03 -->

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d40de5ce9d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +5484 to +5488
if (this.ownedAttemptByTaskId.get(plan.taskId)?.attemptId !== launchAttemptId) {
this.beginOwnedTaskAttempt(plan.taskId, "launch", {
attemptId: launchAttemptId,
receiptEligible: launchReceiptEligible,
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Refuse launches whose reserved attempt changed

With XUM_ALLOW_MULTIPLE_INSTANCES=1, another backend can recover this process's starting row back to queued and reserve it again with a new attempt ID before startReservedAgentTask reaches this block. Adopting that new ID here makes both backends own and dispatch the same attempt; compare the persisted ID with plan.attemptId and abandon the stale launch instead of taking ownership of the replacement.

AGENTS.md reference: AGENTS.md:L106-L106

Useful? React with 👍 / 👎.

Comment on lines +13441 to +13444
this.beginOwnedTaskAttempt(workspaceId, "reawaken", {
attemptId,
receiptEligible: committedProven,
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Settle reawakened attempts when their send fails

When an interrupted task is reawakened but the subsequent sendMessage/resumeStream fails or starts no turn, WorkspaceService restores the persisted status to interrupted, but nothing settles the fresh owner installed here. readAttemptOutcome therefore returns indeterminate, and the next retry inherits unproven lineage even though no execution survived; the rollback path needs to close and settle this exact attempt.

AGENTS.md reference: AGENTS.md:L167-L167

Useful? React with 👍 / 👎.

Comment on lines +14563 to +14567
const liveExecution =
this.workspaceService.getActiveTurnGeneration(workspaceId) != null ||
this.getWorkspaceTurnManager().getLiveWorkspaceTurnRegistration(workspaceId) != null ||
this.aiService.isStreaming(workspaceId) ||
this.hasPendingAdmissions(workspaceId);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Close admission before sampling terminal-failure activity

If a send reaches onAdmitted after this liveExecution snapshot reports false but before the config updater closes the attempt, the later no-record branch clears only queued work and records the attempt as settled while that newly admitted turn continues running. Close admission synchronously before this snapshot, or take the stop latch under the same serialization boundary, so terminal-no-report evidence cannot race a live successor.

AGENTS.md reference: AGENTS.md:L167-L167

Useful? React with 👍 / 👎.

Comment on lines +3945 to 3949
if (!(await this.rotateAttemptForStartupRedrive(task.id, task))) {
failedAwaitingReportCount += 1;
continue;
}
const followUp = await this.workspaceService.dispatchPendingCompactionFollowUp(task.id);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fence startup compaction follow-ups against the rotated attempt

When startup finds a pending compaction follow-up, this rotation is followed by dispatchPendingCompactionFollowUp, which calls AgentSession.dispatchPendingCompactionFollowUpIfNeeded directly and therefore never invokes admitTaskWorkspaceTurn or carries a TurnAdmissionToken. With XUM_ALLOW_MULTIPLE_INSTANCES=1, another backend can stop, retire, or rotate the fresh attempt after this CAS while the recovered follow-up still enters a turn; route this dispatch through the task-attempt fence or explicitly pass an admission token and staleness probe.

AGENTS.md reference: AGENTS.md:L106-L106

Useful? React with 👍 / 👎.

Comment on lines 2986 to +2990
const settlement = this.attemptSettlementByTaskId.get(taskId);
if (settlement?.attempt === owned) {
// Closed but not yet settled: the producer's awaited write (config, later the receipt) is
// still in flight. Its settlement is the guaranteed next signal (fail closed, not a wait).
if (settlement.phase === "closing") return { kind: "cleanup-pending" };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Match settlement evidence to the persisted attempt

With XUM_ALLOW_MULTIPLE_INSTANCES=1, another backend can rotate the persisted row from attempt A to attempt B while this process retains A in ownedAttemptByTaskId; when A later settles, this branch accepts its in-memory settlement without comparing either ID to entry.workspace.taskAttemptId and returns terminal-no-report for the current, potentially live B attempt. The new admission reread does not cover this classifier, so require the owned and settled IDs to equal the freshly persisted attempt before using them as terminal evidence.

AGENTS.md reference: AGENTS.md:L106-L106

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant