feat(envs): remove core envs from the manifest and their sources from the workspace - #10465
davidfirst wants to merge 336 commits into
Conversation
PR Summary by QodoLoad former core envs as regular registry envs with legacy version pinning
AI Description
Diagram
High-Level Assessment
Files changed (15)
|
Code Review by Qodo
1. Unfixable NonLoadedEnv remediation
|
|
Code review by qodo was updated up to the latest commit c7dd1a7 |
|
Code review by qodo was updated up to the latest commit e6418b9 |
|
Code review by qodo was updated up to the latest commit c9eca3d |
|
Code review by qodo was updated up to the latest commit 3f5c24e |
|
Code review by qodo was updated up to the latest commit 0607c7d |
|
Code review by qodo was updated up to the latest commit b23b273 |
|
Code review by qodo was updated up to the latest commit 94eddce |
|
Code review by qodo was updated up to the latest commit ac4b7d0 |
|
Code review by qodo was updated up to the latest commit ab21e34 |
|
Code review by qodo was updated up to the latest commit 66fd06b |
|
Code review by qodo was updated up to the latest commit 684cdf6 |
|
Code review by qodo was updated up to the latest commit 3df0fcd |
|
Code review by qodo was updated up to the latest commit eedfdcd |
…re without declaring an aspect published before the core envs were removed requires them as phantom dependencies, so they never enter the aspects-graph and harmony injects null for them. require their runtime from the package that requires them.
read the phantom requires off the installed packages of the aspects the workspace loads, and run another install cycle when they are only discovered by the install that installed those aspects. a legacy core env configured with a version is installed by the dependency machinery, so it no longer enters the root policy.
the package name of an aspect follows a different convention for core aspects (and the envs that used to be core, published under it) than for any other component. expose the rule so consumers don't re-implement it.
#10687) Three independent robustness fixes to the workspace compiler, extracted from #10465 so they can land on their own, with unit tests. - `onAspectLoadFail`: skip the recompile cascade when the failing aspect is not a workspace component. Compiling cannot materialize a package that is not installed, but the cascade first imports the aspect's whole dependency closure from the remote and recompiles a large part of the workspace into every injected `node_modules` copy - minutes of work for nothing. The regular missing-aspect handling reports it with a `bit install` remediation instead. - `onAspectLoadFail`: treat `ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING` like the other module-load failures. Node refuses to load a `.ts` file from `node_modules`, which happens when the loaded copy has only sources (e.g. re-created by the package manager mid-install) - compiling the component fixes it. - `buildGroupsToCompile`: filter out env ids that are not nodes on the given graph. Envs installed as packages are not part of the workspace graph, and `successorsSubgraph` throws on an unknown node.
…om-manifest # Conflicts: # .bitmap # pnpm-lock.yaml # scopes/dependencies/pnpm/pnpm.package-manager.ts # scopes/workspace/install/install.main.runtime.ts
…kage (#10711) `teambit.harmony/empty-env` is a core aspect, but it was not in the core-envs list. Because of this, `bit install` handled it as an external env. It added the `@teambit/empty-env` package to the env root and pulled the whole Bit core into the workspace (3187 packages instead of 2). The dependency-resolver and dev-files guards also fetched the env component from the remote on each load. This change adds `empty-env` to `getCoreEnvsIds()`. The same line was added and then reverted in July, before #10535 was merged, with the note "it breaks env loading for the component". I reproduced what the revert saw. In the bit repo itself, `bit show teambit.harmony/empty-env` now reports its own env (`core-aspect-env`) as "not loaded". This is the existing load-groups behavior for every core env in this repo: `bit show teambit.react/react`, `teambit.harmony/node` and `teambit.harmony/aspect` report the same warning on master. The component still compiles and builds. The `bit_pr` job on this PR compiled and snapped `empty-env` with the other 97 dependents. The remove-core-envs branch (#10465) returns only `empty-env` from `getCoreEnvsIds()` and passes full `bit ci pr --build` runs with it. A unit test in the install aspect makes sure that no env package is added to the env root for `empty-env`. It does not assert anything about the other core envs, so it stays valid after #10465.
…om-manifest # Conflicts: # .bitmap # .circleci/config.yml # scopes/envs/envs/environments.main.runtime.ts
…nning bit's dists the dev install re-resolves with a manifest bbit never used; when the two disagree pnpm relinks the whole tree and replaces the workspace components' node_modules dirs, dists included. the install's own post-install compile then dies on a deferred require. also make the core-env e2e assertion follow DEFAULT_ENV instead of naming an env.
the install replaces the node_modules the process is loaded from, so every require it defers past that point throws MODULE_NOT_FOUND - the compiler aspect first, then the pager on the way out - long after the packages themselves have landed.
…alls building them inside an env root fails with node-gyp-build exit 127 since pnpm 12.4.1; nothing in the suite uses them.
…s unloaded their packages depend on this repo's components, whose dists the install just replaced, so the first pass falls back to tsc - whose eager requires pull UI-only packages into the node runtime and break the next bit.
the released bbit installed the current version while the dev binary wanted the pinned one; swapping a package this widely depended on re-keys peer hashes across the tree, so the dev install relinked all of node_modules and replaced the dists bit runs from. reverts the CI scaffolding that worked around the symptom.
… envs" the released bbit resolves core aspects from its own bundle and ignores the policy, so it still installed 1.0.1169 and the whole-tree relink was unchanged (+6152 either way).
…he container nothing in that describe looks at the preview; bundling a react-based env's preview is the tag's memory peak.
the tag is still OOM-killed with the preview task skipped, so the peak is elsewhere in the build - keep the task's coverage.
… relink Temporary, to be reverted. The dev-binary install in bit_pr / check_circular_dependencies relinks the whole tree (6152 packages) instead of the delta (1023 before the merge) and replaces the dists the running bit is loaded from. Everything else on this branch is unchanged, so this isolates the engine as the single variable.
…-install relink" This reverts commit 5db7dff.
… root A phantom require of a legacy core env resolves from the requiring package up to the workspace root, so a package already there satisfies it. Adding the pinned legacy version on top replaced the version the rest of the tree was resolved against: the package manager then re-linked every package whose peer resolution moved with it - the whole tree - re-materializing the workspace components' packages without the dists the running process is loaded from, which crashed the next deferred require with MODULE_NOT_FOUND. The bit repo installs the react env at the root, so it hit exactly this: the install relinked 6280 packages instead of the 1023 delta.
The previous commit guarded only the phantom-require path; the react env reaches the policy through the used-env path, so the downgrade still happened. Move the guard into addLegacyCoreEnvsToPolicy, which both paths funnel through, and make it version-aware: a root that already provides the package at the pinned version or later needs nothing added. These envs are dependencies of the bit package itself, so a bit that still ships them as core aspects installs its own (newer) version at the root. Pinning the older legacy version over it moved what the rest of the tree resolved its peers against and re-linked all 6280 packages, rewriting the dists the running process is loaded from.
Building them inside an env root fails with `node-gyp-build` exited with status 127 since pnpm 12.4.1, so an unrelated native addon decides whether a test can scaffold its fixture. Five suites hit it on the react env alone. Set neverBuiltDependencies once where every e2e workspace is created, and drop the two per-call --disallow-scripts workarounds it replaces.
…writes their packages
Removes the env aspects (
teambit.react/react,teambit.harmony/node,teambit.harmony/aspect,teambit.envs/env,teambit.mdx/mdx,teambit.mdx/readme) from the core manifest to slim Bit. They now act like any other env, installed from the registry.New default env:
teambit.harmony/empty-env(core). A totally empty env - no compiler, no tester, no preview, no dependency policy. Components with no env configured use it and work fully offline out of the box (add → compile no-op → tag/snap → export). Since it has no behavior, it has nothing to drift when bit itself changes - the one env that is safe to keep core (and versionless in models) forever. To get a dev experience, users configure a real env (bit createflows already do).teambit.harmony/aspectandteambit.envs/envare removed like the rest, with zero behavior change. Their implementation is untouched (react-based, preview and all) - users get the exact released behavior afterbit install(the pinned-version machinery auto-installs them). New envs are created from the bitdev env packages (bit create react-envetc.), so these built-in envs are legacy surface. Thebit-aspecttemplate and the harmony starters moved to the core generator aspect, sobit create bit-aspectandbit newkeep working out of the box (the created aspect needsbit installbefore it loads, like any env).Versionless by design. Config entries for the removed env ids are persisted by name, without a version - exactly as they were when core (registered as core-extension names). Keeping them versionless is deliberate on two counts. First, it keeps the env from becoming a dependency edge of its own components; otherwise an env such as react, whose dependency closure includes components that use it as their env, creates circular TS project references and breaks lane/tag builds. Second, it preserves forward compatibility: a re-tag under the new bit keeps the env id versionless, so a teammate who has not upgraded yet (whose bit still ships these as core) can import the re-tagged component and resolve the env - instead of receiving a versioned id their bit has no component for. The alternative (showing the component as modified and pinning the env on the next tag) would silently break not-yet-upgraded consumers.
Backward compatibility. Old components have the removed envs saved without a version.
legacy-core-envs.tsmaps them to pinned versions, applied only at the resolution/loading/install level - stored objects are never mutated. Versionless legacy ids match the env slot ignoring version,bit installauto-adds their packages, and single-instance semantics are enforced (a loaded version is reused rather than loading another copy). Not-installed legacy envs fail fast with aNonLoadedEnvissue suggestingbit install- no scope-capsule isolation in workspace context (which used to take minutes). Old components load without being reported as modified, and re-tagging keeps the env versionless - covered end-to-end bye2e/harmony/legacy-core-env-back-compat.e2e.ts, which imports a component exported by a pre-removal bit (env saved versionless) and asserts it is not modified and stays versionless after a re-tag.Relocated core wiring: the
bit aspectCLI command moved toteambit.workspace/workspace;validateBeforePersistHookmoved toteambit.dependencies/dependency-resolver; the dead@teambit/legacylink is now skipped instead of crashing.Also fixes latent issues this path exposed: versionless seeders filtering out all manifests in
loadExtensionsByManifests, circular env chains causing infinite component-load recursion, versioned core-aspect ids escaping core filters anddoRequiremutating shared core manifests, stack overflows from recursive graph traversal, and a spuriousMissingDistsissue for compiler-less envs.Verified locally: fresh workspace (JS and TS components) - clean status in ~1s, tag/snap/export offline,
bit envs/bit testgraceful; this repo's workspace - status/insights/list-core clean; the seven repo components that relied on the default env are now explicitly set to the node env.bit create <template> --env <removed-env>loads the env's templates on demand from the global scope (pinned version); this path also loads the full manifest graph, and binds manifest deps of legacy envs to their pinned versions (models built when these envs were core don't list them as dependencies). The e2esetCustomEnvhelper installs the env package the fixture imports (e.g.@teambit/node).Also removes the former-core env sources from this repo's workspace (
scopes/harmony/node,scopes/react/react,scopes/harmony/aspect,scopes/envs/env,scopes/mdx/mdx,scopes/docs/readme) - bit now dogfoods them as installed packages like any consumer, and the source-vs-installed duality is gone. Making this pass end-to-end surfaced several general fixes that ride along:.docs.mdximports are detected even when the mdx aspect isn't loaded (latent gap once mdx is no longer core - without it, docs deps silently drop from dependency computation and preview bundling fails).Module._extensionsrequire hooks are restored after each build task. An in-process tester leaves@babel/register's pirates hook installed; the hook claims all.jsfiles (including node_modules, regardless of babelignoreconfig) and breaksrequire()of ESM-only packages in every later task in the process (pirates drops theformatarg node >=22.12 uses to routerequire(esm)).import()instead of a top-level require, immune to the same stale-hook hazard.@bit-no-check; timings manifest covers the split spec files so shard balancing accounts for the heavier env-install suites.