Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/architecture/external-session-import-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ The design does not track source updates or incrementally synchronize an importe
| Obligation | Single authority | Public seam |
| --- | --- | --- |
| Source format, discovery, filtering, paging, decoding, and conversion | Corresponding Storage adapter | `listSessionPage(query)`, `readSession(id)` |
| Shared query, sanitize, and limit contracts | Core external-session | Contracts consumed by adapters and Host |
| Shared query, source Session ID/title/cwd matching, sanitize, and limit contracts | Core external-session | Contracts consumed by adapters and Host |
| Workspace resolution, import concurrency, result classification, staging, publication, and recovery | Runtime Host external-session coordinator | `external-session.catalog.query`, `external-session.import` |
| Current published import count and recent Maka Session IDs | Storage Session authority | `lookupExternalSessionImports(adapterId, sourceSessionIds, limit)`, projected by Host as `importState` |
| Provider admission for stored history | Runtime replay planner | `buildRuntimeEventModelReplayPlan`; continuation has separate admission |
Expand Down Expand Up @@ -94,7 +94,7 @@ The Ledger preserves a source transcript that starts with assistant content. `bu

## 6 · Workspace scope

The Runtime Host TUI external-session surface decides scope. With a current workspace target, it offers current workspace and all, defaulting to current; without a target, it offers only all. If the target disappears before a scoped query, the surface rejects that request instead of silently broadening it. TUI runner forwards this choice rather than deriving scope from the Session driver. External-source scope is independent of the Maka task list's Current/All filter.
The Runtime Host TUI external-session surface decides scope. With a current workspace target, it offers current workspace and all, defaulting to current; without a target, it offers only all. If the target disappears before a scoped query, the surface rejects that request instead of silently broadening it. TUI runner forwards this choice rather than deriving scope from the Session driver. External-source scope is independent of the Maka task list's Current/All filter. The TUI runner briefly coalesces consecutive search edits before querying the Host. Every edit advances the same request revision immediately and retires the displayed rows and cursor, so an older in-flight response cannot repaint or paginate the catalog while the newer query waits for its debounce.

## 7 · Adapter-owned paging

Expand All @@ -104,9 +104,9 @@ Host requests at most `page size + 1` source entries. Each returned entry carrie

## 8 · Codex keyset catalog

Codex keeps no server-side catalog snapshot, SQLite transaction, TTL, or LRU across requests. Cursors are bound to the current query. State DB pages order by `(sort_key DESC, id DESC)`, where `sort_key` is computed once by the query, selected alongside the row, and read straight back off that row to build the cursor — so the position a cursor names is by construction the position the query ordered by. Seconds and milliseconds are normalized into that one numeric key before ordering. The first page reads the newest `state_N.sqlite`; if that generation cannot be read, the page is served by the filesystem fallback rather than by an older generation, because a lower generation is the snapshot frozen at the last bump and is missing everything created since. Continuation names the generation it started on, uses a SQL keyset condition, and stays strict — a missing original generation invalidates the cursor, and a transient read failure remains a persistence failure. Connections close after each request.
Codex keeps no server-side catalog snapshot, SQLite transaction, TTL, or LRU across requests. Cursors are bound to the current query. State DB pages order by `(sort_key DESC, id DESC)`, where `sort_key` is computed once by the query, selected alongside the row, and read straight back off that row to build the cursor — so the position a cursor names is by construction the position the query ordered by. One adapter normalizer accepts finite numeric or numeric-string epoch seconds/milliseconds and parseable date-time strings such as ISO 8601; SQLite ordering, cursor position, and displayed summary timestamps all call that same rule. The first page reads the newest `state_N.sqlite`; if that generation cannot be read, the page is served by the filesystem fallback rather than by an older generation, because a lower generation is the snapshot frozen at the last bump and is missing everything created since. Continuation names the generation it started on, uses a SQL keyset condition, and stays strict — a missing original generation invalidates the cursor, and a transient read failure remains a persistence failure. Connections close after each request.

The filesystem fallback orders by `(mtime DESC, relative path ASC)` across active and optionally archived roots. One traversal has a `maxCatalogCandidates` file bound; exceeding it returns a typed source limit. Stat-known keys reject candidates that cannot enter the current page before reading their bounded heads. The page retains at most `limit + 1` matching summaries instead of materializing the corpus or rescanning it repeatedly for deep pages.
The filesystem fallback orders by `(mtime DESC, fixed-size path identity ASC)` across active and optionally archived roots. Its opaque cursor uses the versioned `f2` filesystem tag; the identity is derived once from the relative rollout path, so deeply nested paths cannot enlarge the cursor past the Host wire bound. One traversal has a `maxCatalogCandidates` file bound; exceeding it returns a typed source limit. Stat-known keys reject candidates that cannot enter the current page before reading their bounded heads. The page retains at most `limit + 1` matching summaries instead of materializing the corpus or rescanning it repeatedly for deep pages.

Keysets resume strictly after the last delivered record. A live source updated between pages may move ahead of the cursor and be absent from that traversal; a fresh catalog query sees the new order. The design does not claim a stable snapshot of mutable external data.

Expand All @@ -118,7 +118,7 @@ Storage counts extant published Sessions whose immutable `externalOrigin` matche

## 10 · Staging, publication, and recovery

Importer validates canonical input before the durable commit attempt. It creates a `transcriptLedgerVersion: 0` staged Session, materializes its Ledger, then publishes it as a usable Session. A pre-materialization failure deletes staging; Host startup `recover()` processes remaining version-0 Sessions. Only published copies count in the catalog. Host coalesces concurrent imports of the same `(adapterId, sourceSessionId)` onto one in-flight Promise. An explicit import after completion creates an independent copy.
Importer validates canonical input and completes deterministic catalog projection before announcing the durable commit attempt. It creates a `transcriptLedgerVersion: 0` staged Session, materializes its Ledger, then publishes it as a usable Session. A pre-materialization failure deletes staging; Host startup `recover()` processes remaining version-0 Sessions. Only published copies count in the catalog. Host coalesces concurrent imports of the same `(adapterId, sourceSessionId)` onto one in-flight Promise. An explicit import after completion creates an independent copy.

## 11 · Unknown outcome and client interaction

Expand Down
10 changes: 5 additions & 5 deletions docs/architecture/external-session-import-design.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ TUI / Desktop 选择来源与外部 Session

## 1 · 模块边界

- `packages/core/external-session` 只定义跨来源 contract、query、sanitize 和 limit 语义,不理解任何来源的文件格式。
- `packages/core/external-session` 定义跨来源 contract、query、来源 Session ID/标题/cwd 匹配、sanitize 和 limit 语义,不理解任何来源的文件格式。
- storage adapter 各自拥有 Claude、Codex、OpenCode 的发现、筛选、分页、解码和消息转换规则。
- Runtime Host 拥有 workspace 解析、wire 边界、导入并发、错误分类、暂存和发布。
- TUI 与 Desktop 只展示 catalog、提交选择、按稳定结果码更新交互,不解析来源数据或错误字符串。
Expand Down Expand Up @@ -152,7 +152,7 @@ Runtime Host TUI 的 external-session surface 是 workspace scope 的唯一决
- 没有 workspace target:surface 只提供“全部”,界面不显示无效的 workspace 切换。
- 如果发起当前 workspace 查询时 target 已失效,surface 明确拒绝,不会省略 workspace 参数后静默查询全部。

TUI runner 只展示 surface 给出的选项并转发用户选择,不再读取 Session driver 自行推导 scope。这个 scope 仍与 Maka Session 列表的 Current/All 标签无关;后者只控制原生 Session 列表的展示范围。
TUI runner 只展示 surface 给出的选项并转发用户选择,不再读取 Session driver 自行推导 scope。这个 scope 仍与 Maka Session 列表的 Current/All 标签无关;后者只控制原生 Session 列表的展示范围。TUI runner 会在查询 Host 前短暂合并连续的搜索输入;每次输入都会立即推进同一个 request revision,并清除当前显示的 rows 与 cursor,因此新查询等待 debounce 时,旧的在途响应既不能回写 catalog,也不能继续旧分页。

## 7 · 分页由 adapter 拥有

Expand All @@ -170,8 +170,8 @@ Codex 不保存跨请求的 SQLite 事务、catalog snapshot、TTL 或 LRU 状

两条来源路径分别使用自己的稳定排序键:

1. **state DB**:`(sort_key DESC, id DESC)`。`sort_key` 由查询算一次、随行一起选出,cursor 直接读回该行上的这个值 —— 这样 cursor 指向的位置必然就是查询排序的位置。秒级与毫秒级时间戳先统一成这一个数值键再排序。首页只读最新的 `state_N.sqlite`;若该 generation 读不了,本页改由 filesystem fallback 回答,而**不是**退到更旧的 generation:更旧那本是上一次跃迁时冻结的快照,跃迁之后新建的会话都不在里面。cursor 记录起始 generation,续页仍只读打开同一个文件,通过 `WHERE` keyset 条件继续,并保持严格 —— 原 generation 已删除或不可读时 cursor 明确失效,读失败仍是 persistence failure。连接用完立即关闭。
2. **filesystem fallback**:`(mtime DESC, relative path ASC)`。一次遍历 active 和可选 archived roots,以 `maxCatalogCandidates` 限制遍历的文件数;超过上限返回 typed limit error。候选先用 stat 已知排序键与当前页尾比较,只有可能进入当前页的候选才读取有界 head 并完成 query/path 校验;内存最多保留 `limit + 1` 个匹配摘要,不物化整个 catalog,也不为深分页重复扫描多轮。
1. **state DB**:`(sort_key DESC, id DESC)`。`sort_key` 由查询算一次、随行一起选出,cursor 直接读回该行上的这个值 —— 这样 cursor 指向的位置必然就是查询排序的位置。adapter 中的唯一 normalizer 接受有限数值或数字字符串形式的 epoch 秒/毫秒,以及 ISO 8601 等可解析 date-time 字符串;SQLite 排序、cursor 位置和展示的摘要时间都调用同一条规则。首页只读最新的 `state_N.sqlite`;若该 generation 读不了,本页改由 filesystem fallback 回答,而**不是**退到更旧的 generation:更旧那本是上一次跃迁时冻结的快照,跃迁之后新建的会话都不在里面。cursor 记录起始 generation,续页仍只读打开同一个文件,通过 `WHERE` keyset 条件继续,并保持严格 —— 原 generation 已删除或不可读时 cursor 明确失效,读失败仍是 persistence failure。连接用完立即关闭。
2. **filesystem fallback**:`(mtime DESC, fixed-size path identity ASC)`。opaque cursor 使用版本化的 `f2` filesystem tag;identity 由相对 rollout path 一次派生,因此深层路径不会使 cursor 超过 Host wire 上限。一次遍历 active 和可选 archived roots,以 `maxCatalogCandidates` 限制遍历的文件数;超过上限返回 typed limit error。候选先用 stat 已知排序键与当前页尾比较,只有可能进入当前页的候选才读取有界 head 并完成 query/path 校验;内存最多保留 `limit + 1` 个匹配摘要,不物化整个 catalog,也不为深分页重复扫描多轮。

keyset 的语义是“继续读取严格排在最后交付项之后的记录”。如果一个尚未读取的 live Session 在两页之间更新并移动到 cursor 之前,本次遍历可能看不到它,但不会因此重复已经交付的行;重新打开或刷新 catalog 会看到当前最新顺序。这是实时可变来源下不持有 snapshot 的明确边界。

Expand All @@ -185,7 +185,7 @@ Host 同时限制每页项目数和编码后的 JSON 字节数。若下一项使

## 10 · 暂存、发布与恢复

导入先创建 `transcriptLedgerVersion: 0` 的暂存 Session。Ledger 物化完成后才升级为已发布状态,并出现在任务列表和 catalog 的副本统计中。
导入在宣布持久化提交开始前,先完成 canonical 输入校验和确定性 catalog 投影。然后创建 `transcriptLedgerVersion: 0` 的暂存 Session。Ledger 物化完成后才升级为已发布状态,并出现在任务列表和 catalog 的副本统计中。

- 物化前失败:删除暂存 Session。
- Host 重启:`recover()` 继续处理版本 0 的 Session。
Expand Down
132 changes: 132 additions & 0 deletions packages/cli/src/__tests__/pi-tui-runner.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ import type { InteractionFormResponse } from '@maka/core/interaction';
import type { SkillInvocationResult } from '@maka/core/skill-invocation';
import type {
AgentGraphClientSnapshot,
ExternalSessionCatalogItem,
TurnMessageSubmitResult,
} from '@maka/runtime-host/protocol';
import { SessionActivityRegistry } from '@maka/runtime/goal-turn-lifecycle';
Expand Down Expand Up @@ -6506,6 +6507,137 @@ Slug openai-work<cursor>
await run;
});

test('coalesces external catalog search while retiring stale responses immediately', async (t) => {
const terminal = new FakeTerminal();
const driver = new SlashCommandDriver([]);
const queries: Array<string | undefined> = [];
const requests: Array<{ text?: string; cursor?: string }> = [];
let resolveStale!: (page: { sessions: ExternalSessionCatalogItem[]; nextCursor: null }) => void;
const externalSessions = {
listScopes: () => ['all'] as const,
listSources: async () => ['codex'],
listSessions: async ({ text, cursor }: { text?: string; cursor?: string }) => {
queries.push(text);
requests.push({ ...(text === undefined ? {} : { text }), ...(cursor ? { cursor } : {}) });
if (text === 'code') {
return new Promise<{ sessions: ExternalSessionCatalogItem[]; nextCursor: null }>(
(resolve) => {
resolveStale = resolve;
},
);
}
return text === undefined
? {
sessions: [
{
id: 'old',
name: 'Old empty-query result',
hostCwd: '/repo',
importState: { importedCount: 0, importedSessionIds: [], isImporting: false },
},
],
nextCursor: 'old-next',
}
: { sessions: [], nextCursor: null };
},
importSession: async () => {
throw new Error('unused');
},
};
const run = runMakaPiTui({
title: 'Maka',
driver,
cwd: '/repo',
model: 'claude-sonnet-4-5',
connectionSlug: 'claude-subscription',
permissionMode: 'ask',
terminal,
externalSessions,
});

terminal.input('/session');
terminal.input('\r');
await waitFor(() => plainTerminalOutput(terminal.output()).includes('Import external session'));
terminal.input('\r');
await waitFor(() => queries.length === 1);

terminal.input('c');
terminal.input('o');
terminal.input('d');
terminal.input('e');
assert.deepEqual(queries, [undefined]);
await waitFor(() => queries.length === 2);
assert.equal(requests[1]?.cursor, undefined);
assert.doesNotMatch(plainTerminalOutput(terminal.screenOutput()), /Old empty-query result/);

t.mock.timers.enable({ apis: ['setTimeout'] });
terminal.input(' ');
t.mock.timers.tick(121);
await new Promise<void>((resolve) => setImmediate(resolve));
t.mock.timers.reset();
assert.deepEqual(queries, [undefined, 'code']);
terminal.input('\x7f');

terminal.input('x');
resolveStale({
sessions: [
{
id: 'stale',
name: 'Stale code result',
hostCwd: '/repo',
importState: { importedCount: 0, importedSessionIds: [], isImporting: false },
},
],
nextCursor: null,
});
await new Promise<void>((resolve) => setImmediate(resolve));
assert.doesNotMatch(plainTerminalOutput(terminal.screenOutput()), /Stale code result/);
await waitFor(() => queries.length === 3);
assert.deepEqual(queries, [undefined, 'code', 'codex']);

terminal.input('\x1b');
exitMaka(terminal);
await run;
});

test('cancels external catalog search timers during runner shutdown', async () => {
const terminal = new FakeTerminal();
const driver = new SlashCommandDriver([]);
let listCalls = 0;
const externalSessions = {
listScopes: () => ['all'] as const,
listSources: async () => ['codex'],
listSessions: async () => {
listCalls += 1;
return { sessions: [], nextCursor: null };
},
importSession: async () => {
throw new Error('unused');
},
};
const run = runMakaPiTui({
title: 'Maka',
driver,
cwd: '/repo',
model: 'claude-sonnet-4-5',
connectionSlug: 'claude-subscription',
permissionMode: 'ask',
terminal,
externalSessions,
});

terminal.input('/session');
terminal.input('\r');
await waitFor(() => plainTerminalOutput(terminal.output()).includes('Import external session'));
terminal.input('\r');
await waitFor(() => listCalls === 1);
terminal.input('x');
exitMaka(terminal);
await run;
await delay(160);
assert.equal(listCalls, 1);
});

test('reports the durable Session id when import succeeds but opening fails', async () => {
const terminal = new FakeTerminal();
const driver = new FailingSwitchSessionDriver([]);
Expand Down
Loading