feat: Phase 2b — ingest and ETL layer

[TheMeinerLP]

Replaces #3, closed. Same content, rebuilt as a single squash commit on clean/phase-2b-ingest because a secret scanner flagged disposable test credentials in the old commit history and history cannot be rewritten in this environment. See the closing comment on #3 for detail.

Phase 2b of Apus: the ingest/ETL layer. Stacked on PR #2 (phase 2a), which must land first.

A WorldSource is now configured once and Apus takes it from there: it polls for new
world data, normalises whatever layout it finds into a versioned World Bundle in S3, and
the render path picks it up without knowing where the world came from.

The contract, proven end to end

The whole point of this layer is that the render path stays ignorant of world origins.
That was a claim until now. IngestRenderContractTest runs the real ingest against a
Bukkit-layout world in real MinIO, then renders the resulting bundle with the phase 1
runner image — and a tile lands in the map bucket. No production code had to change
for the contract to hold.

What this delivers

Component	Role
LayoutDetector	Recognises vanilla, Bukkit and nested layouts; normalises all of them to overworld/the_nether/the_end
BundleWriter	Writes the bundle, manifest last — the commit point that makes half-unpacked worlds impossible without S3 transactions
S3SourceConnector, PterodactylConnector	Fetch raw data; only this step is source-specific
WorldSource/WorldIngest CRDs + reconcilers	Cron-driven polling, job orchestration, retention
ingest container image	Runs the ETL as a Kubernetes Job

Adding a source later costs one connector: discover() and fetch(). Transform and
load are shared.

Security findings fixed in this branch

This is the layer that handles untrusted input — unpacked server backups and
uploaded archives — so the findings here matter more than elsewhere.

• LayoutDetector followed symlinks and did not confine paths to the work root. A
  crafted archive with world/region -> /etc would have put system files into a bundle.
  Paths are now resolved and confined, symlinks rejected, world names validated.
• Archive extraction had no size limits. A zip bomb could fill the node's disk and
  evict unrelated pods. Bounded by total size and entry count, and the job now declares
  ephemeral-storage.
• Signed Pterodactyl download URLs leaked into logs through an error message. They
  are short-lived credentials for an entire server backup.
• A WorldIngest could drive a foreign WorldSource — writing its status and
  deleting its bundles — because only the name was checked. Ownership is now verified by
  name and UID.

Two critical correctness bugs, both caught by review

An ingest could report success while its bundle stayed invisible. The reconciler took
the terminal phase from the pod log, but IngestMain logs phase=Succeeded just before
exiting — before the Job controller sets succeeded. A reconcile landing in that window
marked the resource terminal, so the completion handler never ran: no latestBundle, no
retention, no retry, and a complete bundle sitting in S3 that nothing referenced.
Terminality now comes only from Job status.

Retention could delete another source's live bundle. Bundle paths were keyed by
tenant and world name, so two sources with a world called world — the Minecraft default
— shared a prefix, and pruning crossed the boundary. Paths are now scoped by source, and
every source's latestBundle in the namespace is protected.

Known gaps

• The Pterodactyl response envelope is the one part verified only against community docs,
  not panel source. It is now rejected if it does not match, instead of silently looking
  like "no backups available".
• minecraftVersion comes from a new field on the world selector rather than from
  level.dat — the project carries no NBT dependency by design.

Testing

160 operator tests, 65 ingest tests, plus integration tests against real MinIO and k3s.
Container-based tests stay out of build: ./gradlew :ingest:integrationTest,
./gradlew :runner:integrationTest.

A re-review of the final fix wave is still outstanding — the two critical fixes were
verified directly, the remainder rests on the fix agent's own test evidence.

[gitguardian]

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
35899483	Triggered	Generic High Entropy Secret	99c38e7	ingest/src/main/java/net/onelitefeather/apus/ingest/IngestConfig.java	View secret
35899484	Triggered	Generic High Entropy Secret	99c38e7	operator/src/main/java/net/onelitefeather/apus/operator/ingest/IngestJobBuilder.java	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secrets safely. Learn here the best practices.
3. Revoke and rotate these secrets.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[TheMeinerLP]

Superseded by #19. Rebuilt as a fresh squash on a base where the shell env-var validation already uses the require_env helper (and, for the phase-1 lineage, a docs correction to a plan file's stale code sample), so no follow-up fix commit is needed on top. Closing in favor of #19.