Fix OneDFT gradient task ordering and MPI distribution - #222
Open
DCM-Uni-Paderborn wants to merge 1 commit into
Open
Fix OneDFT gradient task ordering and MPI distribution#222DCM-Uni-Paderborn wants to merge 1 commit into
DCM-Uni-Paderborn wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Problem
OneDFT assembles primitive fields in atom/task order and returns the exchange-correlation energy density in the same point order. The host gradient path subsequently sorted tasks by workload and then only restored their parent-atom order. That does not recover the original order of multiple tasks belonging to the same atom. Consequently, energy-density values could be contracted with partition-weight derivatives at different quadrature points. Energies and potentials were unaffected, while nuclear gradients could be strongly wrong and asymmetric. High-angular-momentum bases exposed the problem particularly clearly because screening produces more heterogeneous task sizes.
The existing device ordering correction in #215 does not fix this host-side intra-atom mismatch and distributed OneDFT gradients were still disabled.
Implementation
The gradient path now attaches every model energy-density block to its task before any workload sorting. For MPI execution, model inputs are gathered and arranged atom by atom as before; model energy densities and point-coordinate gradients are transformed back to rank order and scattered to the corresponding local tasks. Rank-local Pulay, quadrature-weight, point-motion, and direct atomic-coordinate derivatives are then summed collectively. The device path additionally gathers and reorders the unpartitioned quadrature weights needed by the model. Explicit layout checks detect incomplete or inconsistent point consumption.
Validation