Skip to content

fix(web): preserve image-only tool result evidence - #539

Open
627150795 wants to merge 1 commit into
openpi-dev:mainfrom
627150795:codex/issue-471-image-results
Open

627150795 wants to merge 1 commit into
openpi-dev:mainfrom
627150795:codex/issue-471-image-results

Conversation

@627150795

Copy link
Copy Markdown
Contributor

Problem

Closes #471.

An image-only tool result can render as "No output" in the Web transcript, hiding that the tool returned an image. Mixed results and specialized tool cards can also lose that evidence after snapshot reload.

Value

Users can distinguish an unavailable image preview from an empty result or a failed tool call, including when reviewing a saved Session.

Approach

Preserve bounded image metadata and content-omission counts in the existing Web projection. Tool cards display the image count, available MIME type and encoded byte estimate, and an explicit preview-unavailable notice. Text remains visible in mixed results, and isError independently determines failure status. Inspect at most 64 content parts and retain at most 16 image metadata entries; no Base64 payload is sent to the browser. Rebuild the committed Web assets.

Validation

  • Focused Node projection tests: 21 passed, including image-only, mixed, error, and bounded-content cases.
  • Focused Vitest transcript/store tests: 55 passed, including SSE updates and snapshot hydration.
  • Playwright image-result regression: 1 passed; visually checked the unavailable-preview notice before and after reload. This uses a synthetic projected snapshot, not a real provider or image renderer.
  • The original isolated projection reproduction now passes: 2 checks.
  • bun run check and git diff --check: passed.
  • bun run test on Windows / Node 24.14.0: 1,515 passed, 2 failed, 10 skipped. The failures were the existing git-info/process.test.ts 1-second process timeout and workflows/execute.e2e.test.ts detached-workflow settlement timeout, also seen on the audit baseline. The runner stopped before its later Windows serial and Vitest partitions. Full local suite success is not claimed.

Impact

  • User-visible behavior: image evidence and unavailable previews remain visible in generic, file, and activity tool cards, including after reload.
  • Model-visible context/tools: none.
  • Runtime/lifecycle: projection and rendering only; no new background work.
  • Persisted config/data: none; existing Session content remains the source of truth.
  • Compatibility/risk: new projection fields are optional. This does not add image transport or preview rendering; metadata and omission counts describe only the bounded inspected content.

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 2192f52.

The bounded image-evidence projection is a good direction: metadata is capped and sanitized, base64 payloads do not reach the browser, and snapshot/reducer coverage is present. Two acceptance issues remain:

  1. The default collapsed tool row still does not tell the user that the result contains an image. Image count and truncation details live only inside the closed details body, while the summary shows only the tool name and success state. The browser test expands the row before asserting, which masks this default-state gap. Please surface a compact image indicator or count in the collapsed summary.

  2. The Playwright test injects an already-projected snapshot containing images and uses only an empty SSE heartbeat. It therefore does not exercise image tool result -> projectMessage -> tool_execution_end SSE -> UI -> reload. Please add an isolated image fixture that covers this full browser path and verifies that no image payload leaks into serialized HTML or snapshot transport.

Local validation: bun run check passed; 21 focused protocol/evidence Node tests and 55 focused UI tests passed. Hosted CI is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(web): 图片工具结果被静默丢弃并显示为成功但无输出

2 participants