Full join output partitioning - #25554
Open
gstamatakis95 wants to merge 2 commits into
Open
gstamatakis95 wants to merge 2 commits into
gstamatakis95 wants to merge 2 commits into
Conversation
gstamatakis95
marked this pull request as ready for review
September 20, 2026 22:20
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
A partitioned Full Outer Join reports
UnknownPartitioningeven when both inputs are already partitioned the same way on the join keys. Every output row stays in the partition its key was routed to, so the report throws away a real guarantee. A later aggregate or join on the join key then gets aRepartitionExecit does not need.The left key alone cannot describe the output, because right only rows carry NULL in the left key in every partition. The key that agrees with the partition number is
coalesce(left_key, right_key).What changes are included in this PR?
symmetric_join_output_partitioningtakes the join keys and the join's equivalence properties. For a Full join whose inputs pass the existing co-partitioning check, it reports the left partitioning restated onCASE WHEN l IS NOT NULL THEN l ELSE r ENDfor each key pair, throughPartitioning::adapt. Hash keeps its count, Range keeps its split points and sort options. Anything else still reportsUnknownPartitioning.COALESCE(r.k, l.k)matches as well.HashJoinExecin partitioned mode,SortMergeJoinExec, andSymmetricHashJoinExecpass their keys and equivalence properties into the helper.The CASE form is what the planner produces for a two argument
COALESCE, so a parent's expression compares equal structurally.What is the testing strategy for this PR?
joins/utils.rscover the Hash, Range, and multi key outputs, the fallbacks toUnknownPartitioning, projections that keep or drop a key, the sort merge and symmetric hash join paths, and the equivalence of both coalesce orders.enforce_distribution.rstests show no shuffle for a parent join on the coalesced key over Hash and Range inputs, in either key order and over a swapped join, and a shuffle for a parent joon the plain key.range_partitioning.slttests 49 to 53 show plans and results for an aggregate onCOALESCE, a chained Full join, the hash repartitioned variant, the negativeGROUP BY l.range_keycase with its single NULL group, and the reversed coalesce order.UnknownPartitioningfor Full joins now asserts the coalesced Hash partitioning.Are there any user-facing changes?
Plans with a partitioned Full join followed by an operator keyed on
COALESCEof the join keys lose aRepartitionExec. Results do not change.symmetric_join_output_partitioningis crate private, so there is no public API change.