Skip to content

feat: Phase 6 — push sources, upload path and Paper plugin - #16

Closed
TheMeinerLP wants to merge 2 commits into
clean/phase-5b-uifrom
clean/phase-6-push
Closed

feat: Phase 6 — push sources, upload path and Paper plugin#16
TheMeinerLP wants to merge 2 commits into
clean/phase-5b-uifrom
clean/phase-6-push

Conversation

@TheMeinerLP

Copy link
Copy Markdown
Contributor

Replaces #8, closed. Same content, rebuilt as a single squash commit on clean/phase-6-push because a secret scanner flagged disposable test credentials in the old commit history and history cannot be rewritten in this environment. See the closing comment on #8 for detail.

The last phase of the Apus spec. Stacked on PR #7.

Until now Apus pulled worlds — from an S3 bucket or a Pterodactyl panel. This adds the reverse: a running Paper server pushes its own world, and a user can upload one through the UI.

What this delivers

  • paper-worldpush — a Paper plugin that copies a live world consistently (pause autosave, save once, copy, resume) and incrementally, uploading only region files whose mtime or checksum changed. All copying and uploading happens off the server thread; only the save itself touches the main thread, because Bukkit requires it.
  • Push and upload connectors in the ingest module, plus the wiring so those source types actually run.
  • POST /api/uploads with presigned multipart, so world files never travel through the API.
  • POST /api/push/{token} — the one endpoint that authenticates with a tenant-bound service token rather than a user login, because a server plugin must not stop working when a person leaves the team.
  • Push tokens are now provisioned by the operator: cryptographically random, one per tenant, written only into a Secret. Never in status, events or logs — status carries the Secret's name at most.

Upload restrictions, measured rather than claimed

A presigned URL is a transferable credential: what it permits, it permits to anyone holding it. So these were verified empirically against real MinIO rather than asserted.

  • Prefix confinement is structural — the S3 key is a pure function of the server-derived tenant namespace, never client input.
  • Redirecting a presigned part to a different key → 403 SignatureDoesNotMatch.
  • Exceeding the declared part size → 403. This was the one expected to be weak; it is not.
  • Total size cap is enforced at completion via ListParts, never presigned — an oversized upload is aborted and never becomes a readable object.

Caveat stated plainly: verified against MinIO, not independently against Ceph RGW. Same SigV4 mechanism, but that is an inference, not a second measurement.

An unresolved design conflict, stated rather than hidden

The plugin pushes many individual raw region files incrementally. The staged connector expects one object per version. The HTTP call between them is now correct, but an ingest triggered that way would likely find nothing usable.

The upload path (single archive) works end to end and is proven by PushIngestEndToEndTest against real MinIO. The plugin path needs its own design pass.

This surfaced because the plugin and the API endpoint were built simultaneously in separate worktrees — each side's assumption was reasonable, and neither could see the other's.

Also in this PR

  • The design spec now opens with a "Stand der Umsetzung" section: which phases shipped, that sharding was deliberately not built after its spike, and what is knowingly open.
  • Spec corrections: several modules still said Java 21 where everything is on 25, and several "open points" in section 15 had been resolved during implementation.
  • Fixed a stale assertion in the ingest/render contract test. It listed expected bundle objects exactly and broke once level.dat was legitimately added by the writer. The check now requires every mandatory object and still rejects genuinely unexpected ones, without breaking on spec-documented sidecars.

Known gaps

  • RBAC for the API's token lookup is broader than ideal (cluster-wide Secret read); a narrower path is documented but not implemented. No Helm or Kustomize manifests exist in the repo, so RBAC lives in javadoc only.
  • The plugin's save window — pause autosave, save, copy, resume — is untested; that needs a real Paper server, which spec section 13.2 already concedes.
  • The OIDC flow has still never run against a real broker; none is chosen (spec section 15).

Final phase of the Apus spec, on top of Phase 5b. Until now Apus pulled
worlds — from an S3 bucket or a Pterodactyl panel. This adds the reverse: a
running Paper server pushes its own world, and a user can upload one
through the UI.

- paper-worldpush: a Paper plugin that copies a live world consistently
  (pause autosave, save once, copy, resume) and incrementally, uploading
  only region files whose mtime or checksum changed. All copying and
  uploading happens off the server thread; only the save itself touches
  the main thread, because Bukkit requires it.
- Push and upload connectors in the ingest module, with the wiring so
  those source types actually run.
- POST /api/uploads with presigned multipart, so world files never travel
  through the API; POST /api/push/{token}, the one endpoint authenticating
  with a tenant-bound service token instead of a user login, so a server
  plugin doesn't stop working when a person leaves the team.
- Push tokens are provisioned by the operator: cryptographically random,
  one per tenant, written only into a Secret, never surfaced in status,
  events or logs.
- Upload restrictions verified empirically against real MinIO rather than
  asserted: prefix confinement is structural (S3 key is a pure function of
  the server-derived tenant namespace); redirecting a presigned part to a
  different key returns 403 SignatureDoesNotMatch; exceeding the declared
  part size returns 403; the total size cap is enforced at completion via
  ListParts, never presigned, so an oversized upload never becomes a
  readable object.
- States an unresolved design conflict rather than hiding it: the plugin
  pushes many individual raw region files incrementally, but the staged
  connector expects one object per version, so a push-triggered ingest
  would likely find nothing usable. The upload path (single archive) works
  end to end, proven by PushIngestEndToEndTest against real MinIO; the
  plugin path needs its own design pass.
- Design spec now opens with a "Stand der Umsetzung" section describing
  what shipped, that sharding was deliberately not built after its Phase 4
  spike, and what remains open; corrects stale Java 21 references to 25;
  fixes a stale exact-match assertion in the ingest/render contract test
  that broke once level.dat was legitimately added by the writer.

This branch replaces feat/phase-6-push (PR #8). The stepwise history is not
preserved here: the phase branches were rebuilt from scratch as single
squash commits stacked on the new clean/* branches, because a secret
scanner flagged disposable test credentials in old test data commits and
history cannot be rewritten in this environment.
@TheMeinerLP

Copy link
Copy Markdown
Contributor Author

Superseded by #24. Rebuilt as a fresh squash on a base where the shell env-var validation already uses the require_env helper (and, for the phase-1 lineage, a docs correction to a plan file's stale code sample), so no follow-up fix commit is needed on top. Closing in favor of #24.

@TheMeinerLP TheMeinerLP closed this Aug 9, 2026
TheMeinerLP added a commit that referenced this pull request Aug 9, 2026
Final phase of the Apus spec, on top of Phase 5b. Until now Apus pulled
worlds — from an S3 bucket or a Pterodactyl panel. This adds the reverse: a
running Paper server pushes its own world, and a user can upload one
through the UI.

- paper-worldpush: a Paper plugin that copies a live world consistently
  (pause autosave, save once, copy, resume) and incrementally, uploading
  only region files whose mtime or checksum changed. All copying and
  uploading happens off the server thread; only the save itself touches
  the main thread, because Bukkit requires it.
- Push and upload connectors in the ingest module, with the wiring so
  those source types actually run.
- POST /api/uploads with presigned multipart, so world files never travel
  through the API; POST /api/push/{token}, the one endpoint authenticating
  with a tenant-bound service token instead of a user login, so a server
  plugin doesn't stop working when a person leaves the team.
- Push tokens are provisioned by the operator: cryptographically random,
  one per tenant, written only into a Secret, never surfaced in status,
  events or logs.
- Upload restrictions verified empirically against real MinIO rather than
  asserted: prefix confinement is structural (S3 key is a pure function of
  the server-derived tenant namespace); redirecting a presigned part to a
  different key returns 403 SignatureDoesNotMatch; exceeding the declared
  part size returns 403; the total size cap is enforced at completion via
  ListParts, never presigned, so an oversized upload never becomes a
  readable object.
- States an unresolved design conflict rather than hiding it: the plugin
  pushes many individual raw region files incrementally, but the staged
  connector expects one object per version, so a push-triggered ingest
  would likely find nothing usable. The upload path (single archive) works
  end to end, proven by PushIngestEndToEndTest against real MinIO; the
  plugin path needs its own design pass.
- Design spec now opens with a "Stand der Umsetzung" section describing
  what shipped, that sharding was deliberately not built after its Phase 4
  spike, and what remains open; corrects stale Java 21 references to 25;
  fixes a stale exact-match assertion in the ingest/render contract test
  that broke once level.dat was legitimately added by the writer.

This branch replaces clean/phase-6-push (PR #16), stacked on
clean2/phase-5b-ui instead of clean/phase-5b-ui. Same content otherwise.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant