v0.8.48: prewarm, tin pg extensions, knowledge search speedups, ui consolidation, keyword search speedups - #8096
Merged
Merged
Conversation
* docs(blog): update enterprise * Pi Babysit: address PR #8002 feedback * Change author in enterprise blog index Updated author name from 'andrew' to 'vik'. --------- Co-authored-by: Sim Pi Agent <pi@sim.ai> Co-authored-by: Waleed <walif6@gmail.com>
… from task failures (#8004) * fix(file-search): skip base64-dominated files and redact query errors from task failures * fix(file-search): count wrapped base64 blocks and verify trigram estimate against pg_trgm
… MCP (#7997) * feat(files): workspace file version history over the v2 API, CLI, and MCP * fix(files): address review feedback on file version history * chore(db): regenerate file version migration after staging's 0364 * improvement(files): scope version rows to the write's workspace and narrow stored provenance status * fix(files): release purged file history atomically with restore * fix(files): release file history through the outbox after the current object is gone * fix(files): release purged history inside the file-row purge transaction * fix(files): resolve metadata version from the record's own storage key * fix(files): never pair a stale file record with a newer version number * improvement(files): simplify file version history internals - read metadata and its current version in one statement; drop the retry and 409 - move provenance policy branching into the provenance module - project stored provenance out of list, head, and get reads; revert reads it on demand - chunk storage-cleanup enqueues to the outbox bulk limit in one place - drop the write-only content_updated_at version column (unreleased 0365) - reuse findCause, the shared cleanup batch constants, and the version-number primitives - share the v2 text presenter between the file and version routes
…8006) * improvement(knowledge): remember a caller's saturated search reach A caller whose tokens reach more documents than the permitted-set limit paid the reach count on every search only to learn again that the set is unbounded. That answer is now remembered per bases and token set for five minutes. The probe now reports saturation apart from a timeout, and only saturation is remembered; an unbounded set only means the legs apply the full access predicate per candidate, so a stale answer costs speed, never access. * improvement(knowledge): keep the permitted-set resolver's TSDoc on its function
) * fix(knowledge): keep Slack searchable while its member crawl runs Members-mode search shows a document only while its member observation is younger than a day, but a full Slack listing re-fetched every thread and so took far longer than a day on a large workspace, leaving most of Slack invisible. Access is now renewed per channel the member can still read, and listings re-read only threads whose root changed, are active, or are due in a rolling 28-day refresh. * fix(knowledge): keep scope renewal due when a channel listing is cut off A Slack conversation listing that stops at its page cap now reports itself incomplete, and the member's renewal watermark only advances once every reachable channel has been listed and renewed. * fix(knowledge): restart listings for Full resync and reset renewal on identity change A Full resync now starts a new full-sync listing even when the connector does not rehydrate, so Slack rereads every thread instead of resuming an ordinary cursor. A member whose identity changes or whose token is rejected also loses its scope-renewal watermark, so renewal runs for the new identity right away. * fix(knowledge): resume member scope renewal across runs Accessible scopes are now listed page by page, and a renewal that does not finish within its budget stores its channel-listing cursor and start time, so the next run continues from there instead of re-reading the first pages and never reaching channels past them. The watermark records when the whole pass began, and an expired cursor restarts the pass once. * fix(knowledge): batch scope renewal and keep refreshed metadata Scope renewal now gathers container pages into batches before scanning the member's stale observations, so the scan runs once per batch instead of once per source page, and an unfinished batch resumes from where it was read. Content that hydrates unchanged under a new hash also refreshes its source URL, modified time and tags, and the member sync log records how many observations renewal kept fresh.
) A member who reaches more of an organization search index than an exact ranking can afford searched keywords through the GIN projection, which scores every chunk matching the term before access is checked, so a common word cost seconds. Where the database provides the tin extension, a BM25 projection of search-index chunks is kept by embedding and knowledge-base triggers, and the keyword leg ranks with Tin first and checks access only on the top of that ranking, widening the window while too few ranked chunks are readable and leaving a page to GIN if the widest window cannot fill it. The query is analyzed by the same websearch_to_tsquery as the GIN path and translated to TINQL; shapes TINQL cannot express keep GIN. The path is gated by the knowledge-tin-keyword flag, a valid Tin index (built only after the backfill completes), and every base being a search index. Script migration 0019 installs the projection only where tin is available and creatable, so self-hosted databases keep an empty table and the GIN path.
* fix(files): bound file version retention per run and document both coalescing cutoffs * fix(files): agree on the current version when a write skipped recording, and tidy CLI version output * fix(files): size each retention batch to the remaining run allowance * fix(files): read file versions against one snapshot of the file
* fix(slack-search): reuse manifest installation in setup * fix(slack-search): register token connection in operation test
* refactor(ui): reuse EMCN icons for simple product glyphs * docs(emcn): add CircleStop usage example --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local>
Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local>
Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local>
…8019) * feat(google-calendar): add RSVP operation to respond to invitations * fix(google-calendar): honor cancellation on RSVP write and tighten outputs * fix(google-calendar): verify the RSVP note was saved
Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local>
The unauthenticated branch of GET /api/auth/sso/providers returned the domain of every configured provider. Sign-in resolves one address at a time through /api/auth/sso/resolve, so the list now answers 401 without a session. Resolve drops the unused providerType field.
Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local>
…WORD (#8035) * fix(self-hosting): stop publishing Postgres and require POSTGRES_PASSWORD The Compose files published the db service on every host interface, and the production file fell back to the password postgres when POSTGRES_PASSWORD was unset. The db is now reachable only over the Compose network, and the production file refuses to start without POSTGRES_PASSWORD. sim-setup writes the value an install needs before bringing it up: a generated password for a new database, and the legacy one for a volume created before it was required, since Postgres ignores POSTGRES_PASSWORD on an existing data directory. * fix(sim-setup): persist a shell POSTGRES_PASSWORD and rotate the effective role A password exported only in the shell is now written to .env, so a later run without the export does not fall back to the legacy value. Rotation steps use the install's POSTGRES_USER and ALTER ROLE CURRENT_USER. * fix(sim-setup): refuse an ambiguous or unstorable shell POSTGRES_PASSWORD An empty export, one that differs from .env, or one .env cannot hold verbatim now stops setup with instructions instead of silently picking a value the database may not have been created with. * fix(sim-setup): only persist URL-safe shell passwords; honor an empty POSTGRES_USER export DATABASE_URL embeds the password unescaped, so a shell-only password is persisted only when it is made of URL-unreserved characters. An empty POSTGRES_USER export resolves to the Compose default, matching ${POSTGRES_USER:-postgres}. * fix(sim-setup): leave a shell-exported POSTGRES_PASSWORD to the operator Compose interpolates a shell value over .env, so the wizard no longer copies it into .env and no longer needs rules for values .env or DATABASE_URL would change. An empty export is still refused, since Compose would use it and refuse to start. * docs(self-hosting): note that POSTGRES_* values must be URL-safe docker-compose.prod.yml composes DATABASE_URL from them as written, so a value containing a URL delimiter initializes the database but breaks the connection string.
* fix(realtime): validate cursor and selection presence payloads The workflow cursor-update and selection-update handlers stored whatever object a client sent into the shared room presence hash and rebroadcast it to every peer, with no shape or size check. An authenticated user with read access to any workflow could park a multi-megabyte blob in shared Redis on every socket they opened and have the server fan it out on each presence broadcast. Both payloads are now rebuilt from a fixed field set before they reach room state or any broadcast, so unexpected keys cannot ride along - mirroring normalizeCellSelection in the table presence handler. Adds a defensive per-field length cap in updateUserActivity so future presence-bearing events inherit the bound, and marks UserPresence.cursor nullable to match the cleared-cursor value the client already sends. * fix(realtime): measure presence field cap in utf-8 bytes The cap compared UTF-16 code units against a byte budget, so a multi-byte payload could pass the check and still land several times larger in the room hash. It now measures the UTF-8 bytes Redis actually stores. Raises the ceiling to 16384. A table cell selection carries four ids capped at 200 characters each, and multi-byte characters plus JSON escaping can expand a legitimate worst case to roughly 5 KB - above the previous 4096, so the old bound could have dropped real presence.
…d tool route (#8042) Tool results reach the model over two lanes, but the model-facing projection was applied on only one. The resume lane runs results through getToolCallTerminalData, which reduces generate_api_key to its status message. The in-band route (POST /api/copilot/tools/execute) returned the handler output verbatim, so the freshly minted plaintext workspace API key crossed to the model and into the turn transcript. The egress projection cannot cover this: its registry is a catalog of pre-existing environment and credential secrets, built once per turn, so a key minted mid-turn is invisible to it. Apply toolResultForModel at the route so both lanes return the same model-facing projection. It is an identity for every other tool.
* feat(memory): preserve durable Agent tool history * docs(agent): explain durable tool history and context limits * feat(memory): bound durable context and retrieve retained tool detail * fix(memory): validate checkpoint recovery and bounded summary coverage * chore(db): format durable memory migration metadata * docs(memory): remove redundant internal README * fix(memory): preserve tool loops and harden durable history * fix(memory): preserve stream usage and bound portable history * fix(memory): surface bounded history and validate replay inputs * fix(memory): retain legacy function call exchanges * fix(memory): admit only complete stored tool exchanges
…#8044) * improvement(copilot): refuse approval-gated tools on the in-band lane Copilot's approval gate is scaffolding today: COPILOT_TOOL_PERMISSIONS_ENABLED is off by default, so nothing is gated on any lane. It is built only on the dispatch lane, which holds a call against a streaming context and a decision row and then declines to dispatch anything the mothership marks in-band. Those calls run via POST /api/copilot/tools/execute, which has no context and no waiter, so turning the flag on would gate the foreground and leave background lanes ungated — a gate that looks enforced but is not. Add toolRequiresApprovalLane next to toolCallNeedsApproval so the covered tool set is defined once, and refuse a gated tool at the in-band route before it runs. Refuse rather than block: a background lane must never hang on a prompt with no row behind it. The check deliberately ignores the stored auto-allow list — an auto-allowed tool sent to the checkpoint lane is admitted there without prompting anyone, so reading it here would only add a database read to reach the same place. Inert while the flag is off, which is the state this ships in; a test pins that. Also record on the flag itself that the gate is a property of the lane, since that is what the next person reads before enabling it. * improvement(copilot): move the approval-lane predicate beside the tool router Importing the dispatch gate module for a one-line predicate pulled the permission persistence layer in with it, whose module body opens a pub/sub channel — two Redis clients and a channel subscription — in every process that loads the in-band route. Move toolRequiresApprovalLane to tool-executor/router.ts, which imports only the catalog. The route already imported @/lib/copilot/tool-executor for ensureHandlersRegistered, so the guard now costs no new import edge at all. The dispatch gate keeps a pointer to it. Its flag-and-catalog behavior is covered in the router tests against the real flag and the real catalog; the route tests keep to what the route does with the answer.
Extend the Confluence attachment allowlist so .pptx and .xlsx files on synced pages and blog posts are listed and handed to the shared parser pipeline the same way PDF and Word attachments already are. Macro-enabled, template, legacy binary and OpenDocument variants stay excluded. Add listing, hydration, genuine-bytes roundtrip and renamed-to-unsupported coverage, and update the connector guides to name the new formats.
* fix(ui): align fallback model subblock styling * fix(ui): keep editor pickers on shared combobox styling
Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local>
Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local>
* improvement(ui): reuse chip inputs for remaining settings fields * improvement(ui): reuse the shared branding upload drop zone (#8031) Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local>
… extension (#8039) * fix(knowledge): retry the Tin projection after a database refuses the extension A script migration can now defer: `up` throws `ScriptMigrationDeferred`, the runner logs it, leaves the name unrecorded and runs the remaining migrations. The Tin projection defers when the database offers `tin` but refuses to create it, so the upgrade after the extension is permitted installs it instead of skipping it forever. * fix(db): keep a deferred Tin projection from failing a direct run `db:push` runs the migration file directly, where a deferral has no migration record to leave unwritten, so a refused extension aborted the push. The direct entry point now adopts the projection where the database allows it and logs the refusal otherwise, while the registered runner still sees the deferral. Also covers continuation after a deferral with a synthetic migration list, which the registry cannot express while the only deferring migration is its last entry.
…earch (#8040) * improvement(knowledge): resolve connectors once and rank each source in its own index Two costs dominated organization search, and both came from deriving per candidate what is true of a whole source. Connector state — deletion, archival, a pending access rewrite, the organization's integration approval, the access mode — is a fact about a connector, so a search resolves it once and filters candidates by the resulting ids. For a member reading ~25k documents, the candidate predicate's connector lookup drops from one per document examined to one per document of a source that still needs this request's live proof: 24,571 evaluations to 610, and the access check from ~390ms to ~160ms. Ranking then follows sources. pgvector post-filters, so one walk over every source spends its scan budget on the sources a caller cannot read: measured recall for a member reading half an index ranged from 0.00 to 1.00, averaging 0.80 over nine queries, two of which returned none of the exact page. A member of a source reads essentially all of it, so that source is walked through an index covering it alone, built after a sync grows it past the threshold and dropped with the connector. Every other source is sliced — mirrored permissions give a caller their own mail, their own files — and those slices are ranked exactly in one statement. Recall over the same queries rises to 0.95 with none below 0.75. Retrieval never waits on an index existing: a source without one is ranked exactly, so a build that is skipped, fails, or has not happened yet costs recall nothing. * improvement(knowledge): resolve the caller's member identities with the rest of the plan A members-mode document is readable while one of the caller's active members observes it, freshly, and which members those are is a fact about the caller. Resolving them with the connectors — one query, one plan — turns each candidate's check into a lookup on the observation key instead of a join to the member behind it, and lets the vector planner read the sources the caller belongs to from the same resolution rather than asking again. * chore(knowledge): drop the stage the access plan made unnecessary * improvement(knowledge): resolve live source grants once per search - ask a live-proof source for the caller's grants once, before either leg, and only when the scope actually reads one, instead of per candidate page - drop the per-page source exclusion and refill loop that followed from the per-page proof - read search result metadata under the stored predicate the rows already passed - end candidate paging on a short page whether or not the leg reorders its results - batch the embedding_search.connector_id backfill in independently committed keyset pages, matching the other projection backfills * test(db): register the projection-source script migration in the push and registry expectations * fix(knowledge): walk a saturated slice and re-read connector state at hydration - fall back to a graph walk when the sliced sources hold more readable documents than one exact ranking may enumerate, since that enumeration has no order and would otherwise rank an arbitrary subset - read content under the full predicate, which re-reads each connector's own lifecycle and approval, so a source deleted, archived or unapproved mid-search stops answering at the gate that returns content * improvement(knowledge): ask a live source for grants only when one of its candidates is read Resolving every gated source's grants up front charged a search for sources it never ranked. The grants are now resolved on first need — when a page of ranked candidates actually contains one of those sources — and memoized for the rest of the search, so a scope that ranks none never asks and one that ranks many asks once. * fix(knowledge): rank uploads without a sliced source and bound the grant resolution - always run the slice that carries connectorless uploads; a caller who is a member of every indexed source had no sliced source beside them, and their uploaded documents were never ranked - resolve live source grants in the provider's bounded pages, so a scope holding more gated sources than one page no longer fails - drop a source's index only while it is invalid, so overlapping syncs cannot have one remove the index the other just built - fan a document's new source out to the chunks a search can reach, which the document lookup index covers, and take the source again when a chunk is enabled * fix(knowledge): refill after a denied gated source, reserve the index build's session, lock the backfill's documents - once a live proof shows the caller does not hold a gated source, rebuild the candidate pages without it, so its candidates no longer hold the slots of sources the caller can read - run the source index build on one reserved connection, so its memory setting and its reset apply to the session that builds - share-lock a page's documents before copying their source, so a detachment in flight waits for the page and then fans its own change out through the trigger * fix(knowledge): start the scan budget over when the pages are rebuilt The candidates already seen before a denied source was known are skipped when the pages are rebuilt, but they still counted against the scan budget, so a refill could exhaust it before reaching the readable candidates ranked after the denied source. * fix(knowledge): serialize the projection source with its document - the set trigger reads the document under FOR SHARE, so a projection write and a document's connector change cannot interleave and leave the older value - the backfill writes a chunk's source only for the document it read that source from; a chunk moved meanwhile is left to its new document's trigger * improvement(knowledge): choose the vector plan by the caller's reach - a caller who reaches at least a quarter of the bases' documents walks the whole graph once, as before: pgvector post-filters, so at that share almost every neighbour the graph visits is theirs and one walk is the cheapest exact answer; searching each source on its own only helps a caller whose reach is a small share of the index - the share is counted once against that bound when the probe saturates, and remembered with the saturation - a bounded caller who is a member of an indexed source walks that source instead of ranking it exactly with the rest * fix(knowledge): search a broad caller's sources when their whole-graph walk comes back short Reach is counted from token overlap, which every readable document has but which connector state, requirement clauses or observations can still refuse. A caller the count called broad whose walk then underfills was not broad: their sources are searched on their own instead, so the misjudgement costs one walk rather than their neighbours. * fix(knowledge): read result metadata under the scope the results were read under Retrieval reports the scope its content was read under — with the live grants it resolved, when a gated source's candidate was read — and the metadata lookup applies the full read predicate under that scope. A gated source's result keeps its name and URL; a revoked one loses them here as it did at hydration. * fix(knowledge): keep what a short broad walk found when searching its sources runs out of budget The per-source search adds to the walk's candidates rather than replacing them, and if it is cancelled by the leg's budget the walk's candidates still stand. * fix(api): fail a v1 knowledge search whose retrieval ran out of time v1 cannot express an incomplete search, so a leg that reached its deadline fails the request, as it did before the route read the retrieval result directly. * improvement(knowledge): mirror each chunk's source and ACL onto the ranking projections - embedding_search and embedding_keyword_tin carry connector_id and acl from their document, set by each projection's own trigger under a share lock and fanned out when a document changes hands or its ACL is rewritten - script migration 0021 installs both, backfills both projections in independently committed keyset pages, and builds a partial GIN index on each projection's acl concurrently - projectionCandidateAccessCondition decides candidate readability on the projection row alone, admitting a superset of the per-row document predicate; a real-database test asserts it never refuses what that predicate admits * improvement(knowledge): decide candidate readability on the ranking row Once a search has resolved its connectors, the vector walks and the Tin keyword window test the caller's access on the row they score — the source and ACL are mirrored there — instead of joining document per visited or ranked chunk. The graph is no longer stalled by a document lookup per candidate, and a keyword window of mostly unreadable chunks costs an array test per row. Content is still read under the full predicate at hydration. * improvement(knowledge): rank the sliced sources on the projection row The sliced sources' readable chunks are enumerated by the projection's ACL index and ranked exactly in one statement, with the saturation bound counted in the same statement; the per-document enumeration through document is gone from the per-source path. * improvement(knowledge): let a resolved scope's reach choose its plan without enumerating documents - with readability decided on the projection row, a resolved scope never needs its readable documents enumerated ahead of ranking: its reach alone chooses between one walk over the whole graph and a search of each source, and is remembered per bases and tokens - the reach share's denominator is the planner's estimate of the bases' documents rather than a count of them - a resolved scope's keyword leg ranks with Tin and decides readability on the ranked row whether or not its reach is bounded * improvement(knowledge): walk every source a caller reads whole; one keyword window for a resolved scope - a source earns its own vector index at a thousand documents: once readability is decided on the row a walk costs a few milliseconds whatever the source's size, where ranking it exactly grows with it - a resolved scope's keyword page takes one wide Tin window and leaves a short page short, rather than widening through narrower windows and then ranking every match * improvement(knowledge): widen a broad reader's short walk; keyword windows follow reach - a broad reader whose whole-graph walk comes back short walks again with a wider scan instead of searching each source: their own sources are already in the walk, and enumerating their readable chunks is a bitmap over most of the index - a broad reader's keyword page widens through the narrow windows as before; only a narrow reader takes the wide window alone * improvement(knowledge): index the projection by source; rank a narrow reader's keyword window once - a partial btree on embedding_search.connector_id lets the planner lead with a few sources where the caller's tokens alone would match most of the index, which is what made a small slice cost a bitmap over all of it - a narrow reader's keyword page returns several pages' worth of readable rows from one wide ranking, instead of ranking the window again for each page * improvement(knowledge): widen a broad walk only when it is short of the result need; a narrower keyword window for narrow readers A walk that found hundreds of candidates needs no rescue; widening it spent the leg's budget and lost them. The wider walk now runs only when the first found fewer candidates than there are results to return. A narrow reader's one keyword window is 20k rows: a few percent of it fills their page several times over, at less than half the widest window's cost. * fix(knowledge): hydrate an oversized ranking page in slices, only as far as the results need A ranking may hand back more candidates than one hydration should read — a narrow reader's keyword window is ranked once for several pages' worth — so a page is drained in slices of the result need and what is left waits, unread, until the results still need it. * fix(knowledge): count a candidate as considered only once its slice is read The slices a refill discards were never read, so the rebuilt pages may hand their readable candidates back. * fix(knowledge): pair an observer with its connector, settle the per-source fan-out, serialize a source's index build - an observation vouches for a document only from a member of the document's own connector: a document that changed hands keeps its old observations, which no longer carry it as a candidate - a source whose search runs out of budget marks the leg partial while the other sources' results stand - a source's index is built under a session lock on the one reserved connection that also builds it, so two syncs cannot interleave the invalidity check and the drop; 0021 drops an invalid leftover before building, and analyzes both projections after the backfill - a broad reader's walk widens when it found fewer candidates than the smallest pool worth reranking * fix(knowledge): give a broad reader's wider walk half of what the leg has left The rerank and hydration of whatever is found — the first walk's candidates at least — keep the rest, so a wider walk that runs out of its share can no longer take the leg's results with it. * improvement(knowledge): let an on-row walk run to its cap, widen a narrow reader's keyword window stepwise, apply scan settings with the deadline - an on-row walk keeps walking, up to a 100k-tuple cap, until its limit is met; the separate wider walk and its diagnostic are gone - the on-row predicate tests the mirrored ACL alone — a member's source is no longer admitted whole, so a document re-owned after its chunk was mirrored is refused at the row - a reach count that ran out of time decides that search only; it is not remembered - a narrow reader ranks the narrowest keyword window first and widens to the wide one only when the page is short; resolved scopes leave the widest window short instead of ranking every match - the HNSW scan settings ride in the deadline statement, one round trip fewer per vector statement * test(knowledge): expect the scan settings to share the deadline statement
…und instead of the deploy (#8058) * fix(knowledge): run the projection source/ACL backfill in the background instead of the deploy Script migration 0021 filled the ranking projections' connector_id and acl columns synchronously, 500 rows per page under a 60s statement timeout. On embedding_search every filled row is re-inserted into each HNSW index, so a page's cost is index maintenance rather than the plan: one page ran past the timeout and the migration, and the deploy, failed. The migration now installs the triggers and builds the indexes only, both idempotent, and the backfill runs from a Trigger.dev task in keyset pages of unfilled rows, paced with a pause between pages and chained across bounded runs. Search does not depend on it: an unfilled row passes the on-row candidate predicate and is decided at hydration under the full document predicate, exactly as every candidate was before the columns existed. Claude-Session: https://claude.ai/code/session_01XU6c7pKRpa5CMoMHKDdqxX * fix(knowledge): decide unfilled projection rows on their document and start the backfill from the outbox An unfilled row no longer passes the on-row candidate predicate outright: it is decided on its document under the resolved candidate predicate, the join per candidate every row paid before the columns existed, so the bounded candidate pools and the exact slice hold only rows that hydration will keep. A partial index on the unfilled rows keeps that branch, and the backfill's keyset pages, an index probe. The migration also leaves one outbox event whose handler starts the backfill task, so it runs after every deploy without an operator. Claude-Session: https://claude.ai/code/session_01XU6c7pKRpa5CMoMHKDdqxX * fix(knowledge): keep the outbox event pending until a deployment without a worker fills the projections Without a Trigger.dev worker the outbox handler no longer detaches the backfill and completes the event; it runs one bounded slice per outbox run and yields with continueOutboxHandler until both projections are filled, so a restart loses at most one slice and the event is never marked done ahead of the work. The index builds run on one reserved connection, so the session-scoped lock timeout covers every build and its reset. Claude-Session: https://claude.ai/code/session_01XU6c7pKRpa5CMoMHKDdqxX * fix(knowledge): start the projection backfill the way the table backfill is started The outbox event and its slice-and-defer handler are gone; nothing else in the repo starts background work that way. The fill is started from the app side as the table backfill is: tasks.trigger on the Trigger.dev worker when one is configured, runDetached in-process otherwise, with the manual script as the operator path after a deploy. The registered migration is now 0022_projection_source_acl_backfill, which supersedes 0021 so a database that already recorded the synchronous shape still gains the unfilled indexes. The page statement timeout is exported so a caller bounding a run can leave it as headroom. Claude-Session: https://claude.ai/code/session_01XU6c7pKRpa5CMoMHKDdqxX * fix(knowledge): refuse a projection backfill page size that is not a positive integer The page size is interpolated into the page statement and a page of nothing would report the projection filled, so a payload that asks for either is refused before the first page rather than quietly reshaped. Claude-Session: https://claude.ai/code/session_01XU6c7pKRpa5CMoMHKDdqxX * fix(knowledge): check the projection backfill budget after the pause between pages A pause that crossed the budget still let the next iteration open a page; the deadline is now checked after the pause, so a bounded run stops before its next page rather than after it. Claude-Session: https://claude.ai/code/session_01XU6c7pKRpa5CMoMHKDdqxX
…he-top-options-compare (#8061) Co-authored-by: Sim Pi Agent <pi@sim.ai>
* docs(library): update openai-vs-n8n-vs-sim * Update apps/sim/content/library/openai-vs-n8n-vs-sim/index.mdx Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: Sim Pi Agent <pi@sim.ai> Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Sim Pi Agent <pi@sim.ai>
…cope's ranking (#8059) * fix(knowledge): apply the source and date filters inside a resolved scope's ranking - a source filter confines the access plan itself, so every on-row predicate, the reach, and the sources the legs walk or rank are that kind of source alone; `upload` keeps only source-less documents - a date filter enumerates the documents it admits off a new `(knowledge_base_id, source_modified_at)` index and ranks them exactly while the planner estimates the window within the probe's limit; a wider window is walked with the date tested through the document, on the row and on the ranked keyword row - a date-bounded set is ranked exactly even when a member source has its own index, since a walk cannot see the date - the reach is keyed and counted by the plan's sources - a keyword leg whose deadline passes before its ranking is resolved is short rather than failed * fix(knowledge): enumerate a small filtered source so both legs rank inside it * chore(db): mark the document date lookup index concurrent in the schema * fix(knowledge): estimate a filter's size under a short deadline of its own
…w, and rerank a person's search (#8060) * improvement(knowledge): page search results, take a custom date window, and rerank a person's search - the Search page asks for 50 chunks, collapses them to one card per document, shows ten at a time and reveals more on request - `updated` gains a custom range: `from`/`to` days in the URL, sent as an inclusive `modifiedAfter`/`modifiedBefore` window; `modifiedBefore` joins the filter contract, the filter intersection, the document conditions, the on-row date test, the filtered-set estimate and the bounded probe - the dashboard search opts into the platform's cross-encoder reranker whenever a Cohere key is configured; reranking stays best-effort * improvement(knowledge): widen a search to 50 chunks only when the reader asks for more * improvement(knowledge): give a filter-first probe the budget its index-driven read can use * improvement(knowledge): keep the default scan for a walk that asks the document per tuple * fix(knowledge): check the window on the request, rerank with any key, and start a refined search at its first page - the filters schema stays a plain object so the Assistant's search input can still extend it; the window's order is checked on the request body - the dashboard asks for reranking outright, and the use case reranks only when the workspace or the platform holds a key - a refinement of the filters starts over at the first page after the reader asked for more - a custom window's days are the reader's local days * fix(knowledge): judge a caller's reranker key by the resolver's policy, and let a custom window be cleared - the credential check tries the platform key before a workspace lookup, and counts a caller's own key only off hosted Sim, as the resolver does - a custom window with no days yet searches nothing and says so; another window drops the days; the picker's Clear reaches the URL through a forwarded `onClear` * test(knowledge): assert only the dropped days, not the URL adapter's next flush * chore(knowledge): narrow the filters once before estimating them * fix(knowledge): wait for both days of a custom window, show the chosen range, and keep the default scan while rows are unfilled - a custom window searches only once both days are chosen; the picker shows the chosen range instead of a fixed label - while the projection still holds rows the backfill has not filled, an on-row walk keeps the default scan cap, since an unfilled row is decided through its document; the answer is read off the unfilled-rows index and remembered for a minute * fix(knowledge): show the filters while a custom window waits for its days
…#8092) * improvement(knowledge): warm the search projection after its backfill * improvement(knowledge): bound the projection warm and contain its extension probe
* improvement(ui): reuse shared product status pages * fix(ui): preserve the existing chat recovery button --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Mac.localdomain>
Co-authored-by: Sim Pi Agent <pi@sim.ai>
Co-authored-by: Sim Pi Agent <pi@sim.ai>
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Contributor
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 757 files
Not reviewed (too large): apps/docs/openapi-v2-files-audit.json (~4,100 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.
Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
Tip: instead of fixing issues one by one fix them all with cubic
Re-trigger cubic
6 tasks
… cleared cursors (#8098) * fix(files,realtime): issue a revision on revert and stop resurrecting cleared cursors Follow-ups to review findings on the v0.8.48 release PR. - POST /api/v2/files/:id/versions/:version/revert now returns the revision naming the content the file holds after the revert. A revert consumes the caller's revision, so without it chaining a second conditional write needed a metadata re-read, and the gap between the two reopened the TOCTOU window the revision exists to close. - The presence roster merge now distinguishes an absent cursor from an explicit null. The fallback is load-bearing: the server rebuilds a socket's presence record on join, so a re-join broadcasts a roster with cursor omitted. But a null is a pointer the peer cleared, and coalescing both with ?? re-pinned a ghost cursor whenever the clearing cursor-update was missed. - Corrected two File block output descriptions that named operations the block does not have: revision claimed a get operation (this block has Get Content, which emits no revision) and lineCount named insert, which is an edit mode. Also collapses the five hand-rolled copies of the revision conditional spread into workspaceFileRevisionField, beside the token it describes. * fix(docs): match the revert example revision to its own record timestamp The revert response example carried a revision token encoding 2026-01-16T09:12:00.000Z while its file and version showed 2026-01-15T10:30:00Z. A revision is base64url(fileId:contentUpdatedAt), so the token has to name the record it ships with; every other example in the document already pairs them, and the one using the later timestamp overrides updatedAt to match. Echoing the old example back as expectedRevision would have named content the response never described.
…mit searches in parallel (#8097) * improvement(knowledge): rank the vector page on the projection and admit searches in parallel * improvement(knowledge): admit before embedding, budget the fill check, pass over emptied slices * improvement(knowledge): keep billing effects awaited and treat a capped pool as exhausted * improvement(knowledge): exclude a denied source through its documents while the projection is unfilled * improvement(knowledge): mark the excluded-sources clause and reset the fill memo in its test
* improvement(knowledge): read the source card with the result and hold Tin readiness longer * improvement(knowledge): keep the Tin readiness hold at a minute
…ve a search's context once, and read identities off the walk (#8100) * improvement(knowledge): score the vector page on the original vectors and pin the clock in the PDF chunk test * improvement(knowledge): score both pages on the original vectors and say so where the walk is scored * improvement(knowledge): resolve a search's context once and carry candidate identities off the walk * improvement(knowledge): one scoped search shape, page and pool cleanups, admission ahead of the embedding beside the scope reads * improvement(knowledge): embed only after every prerequisite holds, refuse a contradicting owner, count the shared fill read * improvement(knowledge): reset the projection-fill memo per integration iteration
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.