adapter.json: Add opt-in deterministic JSON array order - #621
Open
thammel wants to merge 1 commit into
Open
Conversation
Several JSON arrays originate from unordered Python sets: the top-level object lists of an `AbstractObjectStore`, and the set-valued attributes `AssetAdministrationShell.submodel`, `ConceptDescription.is_case_of` and `Extension.refers_to`. Their iteration order varies between runs because of hash randomization, so serializing the same data twice can yield different files. This makes the output unsuitable for diffing, caching and reproducible builds. The new opt-in `sort_arrays` option sorts these arrays by a stable key, while the default behavior stays untouched. Identifiable objects are ordered by their `id`, references by their type, their key chain and their `referred_semantic_id`. Deriving the key from these structural attributes rather than from `str()` or `repr()` keeps the serialized order independent of the representation methods. The key covers every attribute that `Reference.__eq__` considers, because two references differing only in `referred_semantic_id` are distinct set members whose order would otherwise be decided by the set again. The option is also exposed as the encoder classes `SortingAASToJsonEncoder` and `SortingStrippedAASToJsonEncoder`, following the selection scheme of the existing stripped encoders. Like `stripped`, the parameter is ignored when a custom encoder class is given, so an encoder remains the single source of truth for how its arrays are ordered. Fixes eclipse-basyx#573
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Several JSON arrays originate from unordered Python sets: the top-level object lists of an
AbstractObjectStore, and the set-valued attributesAssetAdministrationShell.submodel,ConceptDescription.is_case_ofandExtension.refers_to. Their iteration order varies between runs because of hash randomization, so serializing the same data twice can yield different files. This makes the output unsuitable for diffing, caching and reproducible builds.The new opt-in
sort_arraysoption sorts these arrays by a stable key, while the default behavior stays untouched. Identifiable objects are ordered by theirid, references by their type, their key chain and theirreferred_semantic_id. Deriving the key from these structural attributes rather than fromstr()orrepr()keeps the serialized order independent of the representation methods. The key covers every attribute thatReference.__eq__considers, because two references differing only inreferred_semantic_idare distinct set members whose order would otherwise be decided by the set again.The option is also exposed as the encoder classes
SortingAASToJsonEncoderandSortingStrippedAASToJsonEncoder, following the selection scheme of the existing stripped encoders. Likestripped, the parameter is ignored when a custom encoder class is given, so an encoder remains the single source of truth for how its arrays are ordered.Not covered by this change
write_aas_json_filewithoutsort_arrays.Testing
Seven new test cases in
JsonSerializationDeterministicOrderTestcover the top-level lists,submodel,is_case_of,refers_to, references differing only inreferred_semantic_id, the stripped variant, and the documented "ignored if an encoder is given" contract. They assert the sorted result rather than a specific set order, so they do not depend on set iteration order themselves.Fixes #573