Skip to content

[rush] Dogfood the Rush daemon in rushstack: allow command-scoped plugins, add snapshot workflow and guide - #6046

Open
Sean Larkin (TheLarkInn) wants to merge 9 commits into
mainfrom
thelarkinn-dogfood-rush-daemon
Open

Sean Larkin (TheLarkInn) wants to merge 9 commits into
mainfrom
thelarkinn-dogfood-rush-daemon

Conversation

@TheLarkInn

@TheLarkInn Sean Larkin (TheLarkInn) commented Sep 23, 2026

Copy link
Copy Markdown
Member

Summary

After this change, contributors can build rushstack with the opt-in Rush daemon today, from source, on Linux and Windows. A warm no-op rush-client build is now faster than native Rush on both platforms.

  • Plugins: before this change, every rush-client build in this repo fell back to in-process Rush. The engine rejected any configured plugin, including our rush-published-versions-json-plugin, which is only associated with record-published-versions.
  • Windows: daemon builds also opened a visible terminal window for every tool the daemon started. A slow first start could also leave the daemon permanently unusable until someone deleted a file by hand.

Changes

  • rush-lib: narrower plugin gate. PhasedCommandEngine.parseAsync now uses Rush's own plugin loaders (PluginManager.getPluginsParticipatingInCommand). Configured plugins are allowed only if they are inert for the requested build/rebuild. A plugin is still rejected if it is unassociated (initialized for every command), is associated with the command, or its command-line.json defines the command or one of its phases, or associates a parameter with either. The check fails closed: if a manifest or command-line file can't be read, the request is rejected.
  • rush-lib: fingerprint coverage. captureWorkspaceInputFingerprintAsync now also hashes each configured plugin's autoinstaller package.json and its cached manifest and command-line.json. Changing any of them reloads the generation.
  • rush-daemon: no console windows on Windows. The launcher starts the daemon detached, so it has no console. Windows then gives every console program it starts (Git, tar for cache restores, the cmd.exe shell running each operation) its own visible window. Before serving, the launched daemon now makes windowsHide: true the default for any node:child_process call that doesn't set it. Each hidden child gets a windowless console, and its descendants inherit it. This applies only to the standalone Windows launch path.
  • rush-client-core: startup handoff. The detached startup helper waited for readiness only until the requesting client's 15 s deadline. On a slow first start, such as the first start from a freshly deployed snapshot on Windows, it kept its .starting reservation. Every later client then refused the ready daemon, waited 15 s and fell back to native Rush. The helper now waits for a live launcher for at least 120 s. The fail-closed behavior is unchanged: the reservation is still kept if the launcher exits or never becomes ready.
  • Performance.
    • Client startup: rush-client no longer loads the Rush engine to find rush.json, read daemon settings or parse rushx arguments. Those now live in small modules: RushJsonLocation, RushXCommandLineArguments and DaemonLaunchCommand. Version selection and rushx discovery are loaded only when needed.
    • Runtime fingerprint: it now covers the bundle chunks under dist (which contain the implementation and weren't covered before), and it no longer runs a JavaScript realpath on every file for every request.
    • Project-configuration loads: the daemon's uncached loads resolve rig profiles asynchronously, instead of through heft-config-file's synchronous fallback, which serialized the concurrent loads.
  • Tests. Added:
    • PhasedCommandEngine.test.ts: plugin gate cases
    • a fingerprint test showing that plugin shape files cause a Reload
    • WindowsSubprocessConsoles.test.ts
    • a startup test showing a helper finishing a start that outlasts its client. It fails without the fix with the same "unresolved startup handoff" error users saw.
    • rig-profile tests for the uncached loader, which pass against both the old and the new loader
  • Snapshot tooling. Added the rush deploy scenario common/config/rush/deploy-rush-daemon-dogfood.json. It extracts a self-contained copy of the built client closure into gitignored common/temp/rush-daemon-dogfood.
  • Docs. New contributor guide docs/rush/dogfooding-rush-daemon.md, linked from the root README, the rush-cli-client README and environment-variables.md. The client, daemon and client-core READMEs are updated.

No CI workflow changes, and no daemon block in rush.json: opt-in is RUSH_DAEMON=1, set only for rush-client invocations.

Performance

@rushstack/tree-pattern, one terminal, file caches warm, two identical rounds:

Windows before Windows after Windows native Linux before Linux after Linux native
rush-client daemon status 2.0 s 0.5 s 0.66 s 0.2 s
warm no-op build 4.8 s 2.8 s 3.5 s 2.0 s 1.5 s 1.8 s
daemon start + first build 10.8 s 10.3 s 6.1 s 5.7 s
warm rebuild 15.3 s 13.7 s 14.0 s

A rebuild is dominated by Heft compile time, which the daemon doesn't change. On Windows, one cold rebuild previously opened 22 visible windows and took 85 s; it now opens 0.

Validation

  • Tests:
    • rush-daemon: full suite passes (367 passed, 1 skipped)
    • rush-client-core: full suite passes (77)
    • rush-lib: targeted suites pass (plugin, parser, fingerprint, engine, project configuration and rushx)
    • rush-cli-client: targeted suites pass (routing, launch, version selection and rushx daemon)
  • rush build --to @rushstack/rush-cli-client passes, including lint, with no API report changes. rush change --verify passes.
  • Linux (WSL, Node 22.23.2) and native Windows (fresh C:\workspaces\rdf clone, Node 26.7.0, Git 2.53 for Windows):
    • no fallback message and no native banner
    • the daemon runs from the snapshot
    • graphInitialized: true, with the same PID and generation token across cold, warm, edit-and-rebuild and after-native builds
    • output hashes match a native rush rebuild --only
    • --no-daemon runs native
    • daemon stop leaves no processes
    • the subspace project typescript-v4-test also builds under the daemon
  • On Windows, the first request after a fresh deploy falls back once while the daemon starts; the next request is served, and no reservation is left.

Remaining bottlenecks and open risks

  • Duplicate configuration load. Each warm request loads every project's configuration uncached twice (the lifecycle fingerprint and the engine's pre-iteration check), about 0.5 s on Windows. Removing the duplicate touches rushd: WS3/WS4 integration with Reporter prerequisites #6018's reload-safety design, so it is not in this PR.
  • Slow first start on Windows. The first daemon start from a new snapshot can exceed 15 s while Windows scans the freshly written files. It now self-heals after one native fallback.
  • Windows git hash-object failure. The failure (exit 0xC0000142, seen earlier under Jest) did not occur here, but its cause is unresolved. It may be related to each spawned tool allocating its own console, which this PR removes, but that is not established.
  • Environment identity. The request environment is part of the daemon's identity. A request from another terminal restarts the daemon before running anything.
  • Silent up-to-date builds. An up-to-date warm build prints nothing, because the daemon client has no summary yet.
  • Persistent workers. Persistent Heft/TypeScript workers are not available.

…add daemon dogfooding workflow

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…g guide

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…n starts

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…me fingerprint

The client only needs to locate rush.json, read daemon settings, parse rushx
arguments and build its daemon start command. Move those into small rush-lib and
rush-daemon modules so the client no longer loads the whole engine for every
invocation, and load version selection and rushx discovery only when needed.

The runtime fingerprint now also covers the bundle chunks under dist, which contain
the implementation, and recomputes a resolved path only when a file stat changes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The detached helper waited for readiness only until the requesting client deadline
(15 seconds). A slower first start, such as the first start from a freshly deployed
snapshot on Windows, left its startup reservation behind. Every later client then
refused to use the ready daemon and fell back to native Rush after 15 seconds.

The helper now waits for a live launcher for at least 120 seconds; the reservation
is still retained if the launcher exits or never becomes ready.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…oader

The heft-config-file rig fallback resolves the rig profile synchronously, which
serializes the daemon's concurrent uncached project-configuration loads (two per
request). Resolve it asynchronously first; RigConfig caches the same result, and a
resolution failure still surfaces through a fresh instance exactly as before.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Needs triage

Development

Successfully merging this pull request may close these issues.

1 participant