[BUG][regression] Global _index.json full rewrite is unsafe under concurrent tasks (real corruption under JetBrains multi-agent)

[martin-rueegg]

Summary

Task history persistence uses a global _index.json updated via
read → mutate entire entries array → atomic full-file rewrite (safeWriteJson).
That is safe for a single writer, not for concurrent agents.

We are already seeing production corruption with concurrent agents on the
JetBrains plugin (not observed on last Roo Code CE with serial usage).

This blocks trustworthy parallelizable tasks :
runtime pieces (TaskRegistry, TaskScheduler, TaskSemaphore, status machine)
increase concurrent writers; storage still assumes one writer.

Failure mode

1. Task A reads index (version V).
2. Task B reads index (version V).
3. A writes V+A (atomic rename — OK in isolation).
4. B writes V+B and drops A’s updates.

safeWriteJson prevents half-written JSON; it does not prevent lost updates.

(Real-life situation is probably more complex, as ui_message.json and history_item.json also get corrupted if one opens a task in the UI!)

Observed corruption

• Placeholder task names / wiped fields
• size=0 while task dirs have real UI/ACH data
• Broken delegation graph (parent/child status, delegatedToId, childIds)
• Invalid transitions when one writer restores stale status

Environment

• IDE: PhpStorm 2026.2.1 / WebStorm 2026.2.1 (build WS-262.9437.145)
• Java: 25.0.3
• JCEF Support: Yes
• Zoo Code JetBrains plugin: 3.76.0 (5b0bcd0)
• OS: Windows 11 10.0 (amd64)
• Multiple agents: yes (One session in Phpstorm, another session in Webstorm, they do NOT both need to actually be active)

What changed vs Roo CE (hypothesis)

Zoo added richer lifecycle persistence and parallel-task infrastructure:

• TaskStatus transitions (active/delegated/completed/interrupted)
• More parent/child fields written on delegate/return/cancel
• TaskRegistry / scheduler / semaphore → more concurrent activity

Storage model remained a single global mutable JSON document.

Proposal (phased)

P0 – stop the bleeding

• Optimistic concurrency (occ): record file identity on read (size/mtime/inode);
  refuse write if changed; retry read-patch-write for single-entry updates.
• Process-wide mutex around index updates (same extension host).
• Prefer updating tasks/<id>/history_item.json as source of truth; treat
  _index.json as a rebuildable cache where possible.

Possible implementation: saveFile

P1 – reduce contention

• Workspace-scoped indexes (no single file for all projects).
• Narrow writes: patch one entry, not conceptual “rewrite the world” without merge.

P2 – durable design for #355

• Append-only event log (JSONL or line protocol) or SQLite (WAL) task catalog.
• Repository interface shared by GUI, CLI, and repair tools.

Evidence / tooling

• Repair CLI that rebuilds index from per-task files and detects corruption.
• Snapshot-checked writer prototype (mtime/ctime/size/inode) that turns
  silent clobber into Concurrent modification detected.

Ask

Treat concurrent history writes as a release blocker for parallel tasks,
not a rare edge case. JetBrains multi-agent already proves the race in the wild.