Skip to content

Add opt-in Mongo overlay payload and schema foundation - #519

Draft
sirdeggen wants to merge 18 commits into
mainfrom
codex/overlay-mongo-s02
Draft

Add opt-in Mongo overlay payload and schema foundation#519
sirdeggen wants to merge 18 commits into
mainfrom
codex/overlay-mongo-s02

Conversation

@sirdeggen

@sirdeggen sirdeggen commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Overlay reliability program

This change is one slice of Make Overlays Great Again.

Do not merge until the stacked TypeScript and Go overlay branches have been proven together locally. Isolated CI green is not integration acceptance. Keep this PR draft. No npm publication, live database migration, or Mongo/BASM default activation.

C05 (verified progressive API) is parked. C02 (ts-stack#517) and C04 (ts-stack#518) both edit LookupResolver.ts and must not merge independently.

Keep this pull request in draft until local validation is complete.

Program and scope

  • Tracker or issue: Overlay reliability plan packages S01 (contracts in history) + S02 Mongo foundation
  • Why this change is needed: Overlay-owned Mongo persistence needs versioned schema, payload/GridFS publication, GC serialization, and unknown-commit reconciliation before admission can be atomic.
  • Explicitly out of scope: S03 Engine commitAdmission integration, production Mongo default, Atlas, mixed TS/Go writers
  • Exact head SHA reviewed: fc83b6bb407b0f51720536d879500c6c11e92b88

Impact

  • Public package source or manifest changed; affected packages are listed below
  • Documentation or examples changed

Affected packages: @bsv/overlay (opt-in storage/mongo deep entry; mongodb is an optional peer)

Verification

  • Local commands and results: 59 focused Mongo replica-set tests including 16 MiB/GridFS, staged-upload crash, GC races, unknown-commit, stepdown/SIGKILL, and uint32 index "9" / "4294967295" inserts
  • Hosted CI run: not yet (draft)
  • All applicable checks are terminal and successful on the exact head

Security and dependencies

  • No dependency or lockfile change
  • mongodb driver is an optional peer for the deep entry only

Release and operations

  • No npm publication was performed from a workstation or from this PR
  • Mongo is not the overlay default

Add replica-set schema bootstrap, content-addressed payload publication,
transactionally guarded references/GC, and majority-commit outcome helpers
behind an optional mongodb peer. This is not an AdmissionStorage adapter
and does not change Engine or Knex defaults.
Collection validators used lexicographic $lte on unpadded outputIndex
strings, which rejected legal indexes such as "9". Bound uint32 fields
with $toLong on outputs, consumption edges, GASP nodes, and SHIP/SLAP.
Brand StorageUint64, extract high-complexity helpers, and apply
Sonar-preferred optional chaining, regex, clone, and sort APIs
without changing payload identity or uint32 $toLong validators.
# Conflicts:
#	docs/reference/package-api-migrations.md
#	docs/reference/stack-facts.md
#	governance/repository-health/baselines.json
Merge from main left baselines.json on 2.4.2 while package.json is 2.5.0.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
Health requires overrideRemovalReview.retainedCount to match the live registry.
LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads
over the previous raw-size gates.
Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the
evidence/discovery helper payload growth.
@sonarqubecloud

sonarqubecloud Bot commented Sep 9, 2026

Copy link
Copy Markdown

@sirdeggen sirdeggen added the overlay-reliability Part of org overlay reliability program (project 19) label Sep 9, 2026
@sirdeggen

Copy link
Copy Markdown
Contributor Author

Linked to org project Make Overlays Great Again and tracker https://github.com/bsv-blockchain/make-overlays-great-again/issues/1.

This PR stays draft until the overlay stack is proven locally across ts-stack, go-overlay-services, and go-sdk. Do not merge to main/master to make progress.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

overlay-reliability Part of org overlay reliability program (project 19)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant