Federated divide-run-combine execution for very large analyses

[jaakla]

Split out of #20 (workstream d). Much larger scope than sampling; separated so it does not gate #20.

Problem

For very large analyses, offer suggestions — and an implementation path — for running faster by splitting work across task runners or nodes: a divide / run / combine (map-reduce style) approach over spatial partitions.

Constraints from existing contracts

Two repository rules constrain the design, and both are worth deciding up front rather than discovering late:

• "One canonical implementation creates every declared output." A split/combine runner must not become a second processing implementation. The combine stage has to live inside pipeline.py; a convenience or orchestration entrypoint may wrap it but must not duplicate its processing, QGIS, or report logic.
• The combine must be order-independent. Partition results arriving in a different order must produce byte-identical output, otherwise the project is not reproducible. This is directly provable by the existing input_permutation_invariance metamorphic relation (openmapstack/metamorphic.py) rather than merely asserted — which is the right bar for this feature.

Additional design questions

• Partitioning must be spatially aware. Splitting on row ranges reproduces the coherence problem from Skill guidance for sampled pre-runs and their correctness traps #22: features that must interact end up in different partitions. Partitions need halos, and the combine has to de-duplicate features that appear in more than one halo.
• Hashing. inputs_hash / outputs_hash are canonical file-set hashes. Decide whether partition intermediates participate in the inventory or are explicitly excluded as scratch.
• Run records. One run record for the federated run as a whole, or one per partition plus a parent? The report parity rule ("every declared check appears exactly once") pushes toward one record with per-partition evidence nested inside it.
• Vendor neutrality. Any runner integration (Dask, Sedona, a queue, plain subprocess fan-out) belongs behind an explicit compatibility surface, not baked into the shared contract.

Acceptance criteria

• Federated execution produces byte-identical output to a single-process run of the same project on the same inputs, proven by test, not asserted.
• Order-independence of the combine is proven via input_permutation_invariance.
• No duplication of pipeline logic in the orchestration layer.
• Halo de-duplication is tested with a case where a feature genuinely straddles a partition boundary.

Relationship to #20

Independent. #20's sampled runs make large jobs cheaper to validate; this makes them faster to run. Neither blocks the other.