feat: Phase 2b — ingest and ETL layer - #3

Closed

TheMeinerLP wants to merge 32 commits into

feat/phase-2-operatorfrom

feat/phase-2b-ingest

Commits on Aug 8, 2026

docs: add phase 2b implementation plan for the ingest layer
TheMeinerLP
committed
b5dc775
feat(ingest): add world source and ingest custom resources
TheMeinerLP
committed
db020be
feat(ingest): add world layout detection
TheMeinerLP
committed
b1da672
merge: task 2 — world layout detection
TheMeinerLP
committed
a975a3f
feat(ingest): add bundle manifest and S3 bundle writer
TheMeinerLP
committed
f1c3144
fix(ingest): harden LayoutDetector against path traversal and symlink escapes
TheMeinerLP
committed
1e01f51
merge: task 3 — bundle writer with manifest-last semantics
TheMeinerLP
committed
ea6aff4
feat(ingest): add world source connector interface and S3/Pterodactyl pull sources
TheMeinerLP
committed
a4c2c42
merge: task 4 — connector interface with s3 and pterodactyl sources
TheMeinerLP
committed
3f434b4
build(ingest): add Jackson and Testcontainers MinIO to the version catalog
TheMeinerLP
committed
46ab524
refactor(ingest): replace hand-rolled JSON codecs with Jackson
TheMeinerLP
committed
8834de9
test(ingest): run S3SourceConnectorTest against Testcontainers MinIO
TheMeinerLP
committed
09c0e45
test(ingest): cover PAX path overrides and archive-extraction traversal
TheMeinerLP
committed
8ff846a
fix(ingest): thread source type and Minecraft version through BundleWriter
TheMeinerLP
committed
b83a2aa
feat(ingest): add ingest job entrypoint and container image
TheMeinerLP
committed
c1d2ef4
build(operator): add cron-utils and depend on ingest/AWS SDK for the ingest reconcilers
TheMeinerLP
committed
6cdc718
feat(ingest): add WorldSource and WorldIngest reconcilers
TheMeinerLP
committed
1ab1a89
refactor(ingest): make IngestMain.run public for cross-module integration tests
TheMeinerLP
committed
7e716f1
test(runner): prove the ingest bundle contract with a real render
TheMeinerLP
committed
dbb45e4
fix(ingest): scope bundle paths by source, write level.dat/entities/poi, record true source ref
TheMeinerLP
committed
a4d7fde
fix(ingest): require the owning source's name, add configurable archive limits
TheMeinerLP
committed
6d23131
fix(ingest): bound archive extraction by total size and entry count
TheMeinerLP
committed
33db0b4
fix(ingest): harden PterodactylConnector - timeouts, response validation, no URL leakage
TheMeinerLP
committed
99c4e74
fix(operator): scope bundle store listing/deletion by source name
TheMeinerLP
committed
afcc2c6
fix(operator): trust only Job status for ingest completion, verify source ownership
TheMeinerLP
committed
52b3aae
fix(operator): require exhausted backoff before marking a render terminally failed
TheMeinerLP
committed
16156db
feat(operator): record per-world Minecraft version; bound ingest job ephemeral storage
TheMeinerLP
committed
329da09
docs(ingest): document the source-name and archive-limit env vars
TheMeinerLP
committed
05839f4