th-1fca98: persist a user turn's images so other clients re-render them - #564
Merged
Merged
Conversation
Images attached to send_message rode the live LLM turn only; the inbound message was stored text-only (MessageContent::from_text), so a DIFFERENT client reading the conversation history (desktop viewing a photo the iOS app sent) saw text with no picture — cross-client parity gap. ContentItem now carries an optional `url` and an "image" type; the runner persists each of the turn's image URLs as an image content item alongside the text (persist_message takes an image_urls slice; outbound passes &[]). Text-only turns are byte-for-byte unchanged, and the addition is backward-compatible on the wire (url optional, image additive to the schema enum). Covered by a domain serialization test and an end-to-end runner test that drives a turn with an image and asserts the stored inbound message carries the image content item. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HVAvzYG7unJjCD9c3B9j1f
🦋 Changeset detectedLatest commit: 6df55f4 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A photo uploaded from the iOS Big Smooth app shows locally but not in the desktop app's view of the same conversation — desktop sees text only. Images on
send_messageare consumed for the live LLM turn (with_user_images) but the inbound message is persisted text-only (MessageContent::from_text), so any other client reading the shared conversation history renders text with no picture.Fix
ContentItemgains an optionalurlfield and animage()constructor;typemay now be"text"or"image".MessageContent::from_text_and_imagesbuilds a text item plus oneimageitem per URL.persist_messagetakes animage_urls: &[String]; the inbound user turn passes the turn's image URLs, the outbound reply passes&[].spec/domain/message.schema.jsonContentItemupdated (imageadded to the enum,urlproperty added) to keep the contract in sync.Text-only turns are byte-for-byte unchanged (
from_textstill used when no images). The addition is backward-compatible on the wire:urlis optional (skip_serializing_if),imageis additive to the enum.Scope note: this is the Rust server path that Big Smooth (smooth-daemon) runs on. The polyglot cores (go/ts/python/dotnet) have their own
ContentItem/persist paths; bringing them to parity is a follow-up, tracked separately — Big Smooth does not use them.Verification
smooai-smooth-operatordomain unit test:content_with_images_serializes_image_items_web_readsasserts the exact wire shape clients parse (items[].type/items[].url) and that text-only is unchanged.smooai-smooth-operator-serverend-to-end test:persist_turn_images::inbound_turn_images_are_persisted_for_cross_client_renderdrives a real turn with an attached image against in-memory storage and asserts the stored inbound message carries the image content item.smooai-smooth-operatorlib suite (249 tests) +spec_fixturesgreen; fmt + clippy clean.The matching consumer change (bump the git rev in SmooAI/smooth + render image parts in smooth-web history) lands separately.
🤖 Generated with Claude Code