Fold BraKetSymmetry::Conjugate bra<->ket orientations onto one canonical spelling and eval cache slot - #591
Fold BraKetSymmetry::Conjugate bra<->ket orientations onto one canonical spelling and eval cache slot#591kshitij-05 wants to merge 27 commits into
Conversation
Opt-in (exploit_conjugate): canonicalize_slots reports whether folding the two bra<->ket orientations of a Hermitian (BraKetSymmetry::Conjugate) tensor onto one canonical orientation introduced a conjugation, so eval-node CSE can share one cache slot between the orientations and serve the swapped one via an adjoint. Default off; no behavior change without the opt-in.
Layer the exploit_conjugate byproduct from canonicalize_slots onto the eval tree, so the two bra<->ket orientations of a Conjugate leaf share one cached value. EvalExpr(Tensor, exploit_conjugate) threads the flag into the ToT leaf's canonicalize_slots call and records the conjugation in a new canon_conj() bit. That bit is kept OUT of hash_value() (only canon_phase is folded there), so a Conjugate leaf and its bra<->ket swap hash identically and share a cache slot. binarize(Tensor, opts) then turns the conjugated orientation into an EvalOp::Adjoint over the bare canonical leaf, carrying the SAME canonical index order -- so the existing adjoint evaluator's result(post) = operand(pre).conj() degenerates to a pure elementwise conjugation (post == pre, no transpose) on retrieval. This reuses the tested '+'-adjoint machinery rather than adding a new eval op. Default off leaves every existing path byte-identical: the new binarize branch is skipped and canonicalize_slots is called with exploit_conjugate=false, exactly as before. Test [exploit_conjugate] (replaces the throwaway probe): a proto-indexed Conjugate leaf and its adjoint fold to one hash with exactly one carrying the byproduct; binarize wraps the swapped orientation in EvalOp::Adjoint over the shared bare leaf with matching canonical indices; off by default keeps the two distinct.
Extend the exploit_conjugate conjugation channel to the flat
(protoindex-free) block-canonicalization leaf path, so a flat
BraKetSymmetry::Conjugate tensor and its bra<->ket-swapped partner fold
onto one cached value the same way the ToT/canonicalize_slots path
already does. This is the path flat complex-field Conjugate leaves take.
TensorBlockCanonicalizer::apply() already folds the two bra<->ket
orientations of a Symm tensor (a free relabeling). Factor that
color-based swap into a shared orient_braket_by_color() (apply()'s Symm
branch reuses it, byte-for-byte unchanged) and add
fold_conjugate_braket(), which applies the same swap to a Conjugate
tensor and reports whether it swapped -- for Conjugate the swap carries a
conjugation (C{ket;bra} = conj(C{bra;ket})), so it is a byproduct the
caller must consume, not a free relabeling.
The EvalExpr flat-leaf ctor branch calls it under exploit_conjugate and
records the result in canon_conj_ (kept out of the hash, so the two
orientations share a cache slot); binarize(Tensor)'s existing
EvalOp::Adjoint wrap then serves the swapped orientation as a pure
elementwise conjugation on retrieval, exactly as for the ToT path.
Default off leaves every existing path byte-identical. Limitation:
equal-color bra/ket bundles (identical spaces) are not folded on the flat
path -- that needs a full index-pattern comparison, which only the
bliss/ToT path does; the flat color rule matches apply()'s Symm fold.
Test [exploit_conjugate] gains a flat-leaf section (C{a_1;i_1}:N-C-S)
mirroring the ToT checks: off -> distinct, on -> fold + exactly one
conjugated + Adjoint over the shared bare leaf.
EvalExpr(Tensor)'s canonicalize_slots call passed {} for
named_index_compare in order to reach the exploit_conjugate argument. An
empty comparator is NOT the declared default: canonicalize_slots then
falls back to an internal space()-only lambda, whereas the declared
default (default_idxptr_slottype_lesscompare) orders named indices by
proto-index count first. That proto-count-first order is what lays a
proto-indexed (ToT) coefficient's canon_indices out with occupieds
first -- a layout downstream coefficient-shape detectors rely on. So {}
silently mis-ordered them and broke such consumers.
Pass default_idxptr_slottype_lesscompare{} explicitly, restoring the
comparator every ToT leaf had before the exploit_conjugate arg was
threaded. Flat leaves (block-canon else-branch) are unaffected.
The flat-leaf exploit_conjugate channel folds the two bra<->ket orientations of a Conjugate tensor onto one cache slot, serving the swapped orientation via EvalOp::Adjoint. The ToT (proto-indexed) leaf path lacked the fold: TNV3's canonicalize_slots does not exploit conjugate braket symmetry, so the orientations landed in separate slots. Apply TensorBlockCanonicalizer::fold_conjugate_braket to the ToT leaf before the network canonicalization (the color rule is label-independent and proto-safe) and compose the byproduct into canon_conj_; the ToT TA Result backend already implements adjoint() (conj recurses into nested tiles). Off by default (exploit_conjugate opt-in unchanged). Test tot_conjugate_braket_fold: hashes fold, canon_conj marks the swapped orientation, no behavior change without the opt-in.
There was a problem hiding this comment.
Pull request overview
This PR introduces an opt-in exploit_conjugate path that canonicalizes BraKetSymmetry::Conjugate tensors so bra↔ket-swapped orientations share a single eval/cache identity, recording a conjugation “byproduct” and serving the swapped orientation via EvalOp::Adjoint.
Changes:
- Add
exploit_conjugateplumbing toTensorNetworkV3::canonicalize_slots()and graph construction, plus a network-levelconjbyproduct bit. - Extend
EvalExprwithcanon_conj()(excluded from hashing) and updatebinarize()to wrap conjugated-orientation leaves withEvalOp::Adjointwhen opted in. - Refactor tensor block canonicalization to factor out bra/ket orientation logic and add
fold_conjugate_braket(), with new unit tests covering flat and ToT leaf paths.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/test_tensor_network.cpp | Adds coverage for exploit_conjugate behavior in TN canonicalization metadata and hashing. |
| tests/unit/test_eval_expr.cpp | Adds end-to-end eval/binarize tests asserting leaf folding and Adjoint wrapping behavior when opted in. |
| tests/unit/test_canonicalize.cpp | Adds a ToT-focused regression test validating Conjugate folding behavior and opt-in default-off behavior. |
| SeQuant/core/tensor_network/v3.hpp | Extends canonicalization API/metadata with exploit_conjugate and conj byproduct reporting. |
| SeQuant/core/tensor_network/v3.cpp | Implements Conjugate bra/ket folding in graph coloring and computes the conjugation byproduct. |
| SeQuant/core/tensor_canonicalizer.hpp | Declares fold_conjugate_braket() and an extracted bra/ket orientation helper. |
| SeQuant/core/tensor_canonicalizer.cpp | Implements the extracted orientation logic and Conjugate fold helper; keeps Symm behavior unchanged. |
| SeQuant/core/eval/eval_expr.hpp | Adds EvalExpr(Tensor,bool) doc and canon_conj() API; extends BinarizationOptions. |
| SeQuant/core/eval/eval_expr.cpp | Implements canon_conj_ propagation, passes exploit_conjugate into TN canonicalization, and wraps swapped Conjugate leaves via EvalOp::Adjoint in binarize(). |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| auto md = [&cardinal](const std::wstring& s, bool exploit) { | ||
| TN tn(deserialize(s)); | ||
| return tn.canonicalize_slots(cardinal, nullptr, {}, exploit); | ||
| }; |
| // Opt-in: fold the two bra<->ket orientations of a ToT Conjugate leaf | ||
| // onto one canonical orientation BEFORE the network canonicalization | ||
| // (TNV3 does not exploit conjugate braket symmetry itself), recording | ||
| // the conjugation byproduct exactly like the flat-leaf branch below. |
For a sum whose value the caller asserts to be real, a summand and its adjoint contribute Re(s + s*) = Re(2 s). fold_conjugate_pairs_of_real_sum detects adjoint-related summand pairs via canonical forms (robust to dummy renaming and factor reordering) and folds each pair onto its first member with a doubled scalar. Self-adjoint summands are left untouched. This is the symbolic-layer counterpart of the eval-layer exploit_conjugate channel: it removes conjugate-duplicate terms from the expression itself instead of folding them onto one cache slot at evaluation time.
This reverts commit b284e05.
The eval-node CSE identity (the canonical colored connectivity graph) is value- invariant for ordinary tensors but not for complex tensors whose leaf value is reconstructed downstream (e.g. relativistic Kramers t-amplitudes via an external recon): two contractions can share a canonical graph yet evaluate to genuinely different (NOT merely sign-flipped) tensors, surfacing as a +1/-1 canon_phase collision. Keying canon_phase into the eval-node equality splits those apart. This only ever splits cache entries (never merges), so it is correctness- preserving; real closed-shell paths (all phase +1) are unaffected and lose no CSE. Interim hack. TODO: replace with a faithful conjugation-aware eval-node identity once TensorNetworkV3 exploits BraKetSymmetry::Conjugate (carry the conjugation through canonicalization, the way canon_phase carries the sign).
Two eval nodes that share a canonical graph/leaf but differ in antisymmetric-reorder parity evaluate to negatives of each other (+T vs -T), so they must not share a CSE cache slot. This was enforced by a special-cased canon_phase check in TreeNodeEqualityComparator, flagged as a hack pending a "faithful" fix. Make canon_phase a first-class part of the node identity: fold it into EvalExpr::hash_value() (a no-op for real closed-shell paths, where every phase is +1, so the equality structure is unchanged) and keep the comparator check as the hash-collision guard. This retires the hack and fixes silent CSE over-merging of complex/Kramers contractions (Kramers-restricted CCD now matches the spinor reference).
fold_conjugate_pairs_of_real_sum gains an optional conjugate_op: a map from a summand to an expression the caller asserts to equal the summand's complex conjugate in value. Default remains the algebraic adjoint. A custom map lets callers exploit domain identities that express the conjugate as an index relabeling instead of a bra<->ket swap (e.g. leaf tensors whose label-flipped blocks equal the complex conjugate), so conjugate pairs written in that form are recognized and folded.
Covers the CSV/ToT half of the BraKetSymmetry::Conjugate channel: a proto-indexed (ToT) Conjugate leaf and its bra<->ket-swapped partner canonicalize to the same graph and hash under exploit_conjugate, with canon_conj set on exactly one of the pair, so binarize can serve the swapped orientation through an EvalOp::Adjoint wrapper over the shared cached operand.
Krzmbrzl
left a comment
There was a problem hiding this comment.
I feel like exploitation of conjugate BraKet symmetry should be on-by-default. That would be consistent with how we handle all other symmetries.
Also, I don't think tracking the conjugation result by means of a boolean variable is the right way to go. At some point in the future (ideally already within this PR) we want to handle this to full extent implying that only some tensors might be conjugated whereas others might not. Hence, we need a more granular result telling us exactly which tensors need conjugation. Otherwise, we will have to break the public API eventually to account for this.
I have implemented a TreeIndex class on a different branch that allows uniquely identifying an element in a (potentially nested) (expression) tree. We could use that and then return a vector of TreeIndex objects to specify which tensors need conjugation. This would have the advantage that this interface would still be usable once the canonicalizer can deal with nested expressions.
| /// T{bra;ket} = conj(T{ket;bra}), so folding its two orientations onto one | ||
| /// canonical form carries a conjugation, recorded here (cf. `phase`, which | ||
| /// carries the ±1 linear byproduct of antisymmetric slot reorderings). | ||
| bool conj = false; |
There was a problem hiding this comment.
I feel like a simple boolean flag is insufficient as it might only be a single tensor that needs conjugation whereas others are left unchanged.
There was a problem hiding this comment.
It seems like support for conjugate canonicalization is only added to canonicalize_slots which seems to imply that regular canonicalization still doesn't handle it. If this is true, I think this is inconsistent and should be changed to also support it in regular canonicalization routines.
| // (canonical_bra_ket_bundle_order, v3.cpp above). N.B. metadata.conj is a | ||
| // single network-level bit: rigorous for one Conjugate tensor (the | ||
| // proto-indexed-leaf case that eval-node identities canonicalize), the | ||
| // per-leaf conjugation of a multi-Conjugate-tensor network is future work. |
There was a problem hiding this comment.
is that actually much more complicated? If not, it would make sense to just do the full thing here 🤔
| SlotCanonicalizationMetadata canonicalize_slots( | ||
| const container::vector<std::wstring> &cardinal_tensor_labels = {}, | ||
| const NamedIndexSet *named_indices = nullptr, | ||
| SlotCanonicalizationMetadata::named_index_compare_t named_index_compare = | ||
| default_idxptr_slottype_lesscompare{}); | ||
| default_idxptr_slottype_lesscompare{}, | ||
| bool exploit_conjugate = false); |
There was a problem hiding this comment.
I think it's time to bundle these options into an options struct and pass that instead. The number of parameters starts becoming unwieldy 👀
| auto bra = mutable_bra_range(t); | ||
| auto ket = mutable_ket_range(t); |
There was a problem hiding this comment.
Why does this need mutable_*? It seems like they aren't actually mutated
| for (auto&& idx : bra) bra_spaces.push_back(idx); | ||
| for (auto&& idx : ket) ket_spaces.push_back(idx); |
There was a problem hiding this comment.
Using universal reference is unnecessary if we don't forward the indices
| for (auto&& idx : bra) bra_spaces.push_back(idx); | |
| for (auto&& idx : ket) ket_spaces.push_back(idx); | |
| for (const auto& idx : bra) bra_spaces.push_back(idx); | |
| for (const auto& idx : ket) ket_spaces.push_back(idx); |
| // in via exploit_conjugate. Default keeps Conjugate bra/ket distinctly | ||
| // colored (no fold, no conj), mirroring historical behavior. | ||
| const auto cardinal = TensorCanonicalizer::cardinal_tensor_labels(); | ||
| auto md = [&cardinal](const std::wstring& s, bool exploit) { |
There was a problem hiding this comment.
md seems like a very non-descriptive name. It's short but also doesn't hint at what the function is doing. I'd recommend a more readable/speaking function name
There was a problem hiding this comment.
I think this can be a default-on feature. A few things that concerns me:
- Since this affects the export pipeline (if opted-in), we should include some tests and check if the produced code makes sense.
- Minor: I think the "flat tensor/array" language for proto-index free tensors is conflicting because we refer to them as Tensor of Scalars also. We should stick to one, maybe out of scope of this PR.
| for (const auto &[tensor_ord, bk] : bundle_pos) { | ||
| // bra bundle canonically after ket bundle => canonical form is the | ||
| // bra<->ket-swapped (conjugated) orientation of the input. | ||
| if (bk[0] && bk[1] && *bk[0] > *bk[1]) conj = !conj; |
There was a problem hiding this comment.
I see the comment which says "multi-Conjugate-tensor network is future work". The current version will break if two tensors are conjugated in a network. Agree with Robert that one single bool cannot represent this.
| std::move(sentinel)}; | ||
| } | ||
|
|
||
| // Opt-in: fold the two bra<->ket orientations of a BraKetSymmetry::Conjugate |
There was a problem hiding this comment.
The code block above has very similar structure to this. Maybe factor out the common piece.
| /// | ||
| explicit EvalExpr(Tensor const& tnsr); | ||
| /// \param tnsr The tensor to wrap as a leaf. | ||
| /// \param exploit_conjugate If true, a proto-indexed (ToT) leaf is |
There was a problem hiding this comment.
I assume this line is wrong? The logic is supported for non-ToT case also right?
| } | ||
| } | ||
|
|
||
| // B2: the opt-in exploit_conjugate path folds the two bra<->ket orientations of |
- orient_braket_by_color: read-only bundle access (drop mutable_* ranges
and universal references; the reorientation goes through _swap_bra_ket)
- test_tensor_network: pass default_idxptr_slottype_lesscompare{}
explicitly instead of {} (an empty std::function silently selects
canonicalize_slots' space-only fallback ordering -- a different code
path than real callers exercise) and give the helper lambda a
descriptive name
- test_eval_expr: drop unused <iostream>, drop the 'B2' plan-reference
from a comment
Mirror Variable and Power: Tensor gains a conjugated_ marker -- in the
hash (contributing only when set, so unconjugated tensors hash
identically to before), in static_equal and static_less_than (T orders
before conj(T)), rendered as ^* on the label in to_latex and in the v1
serializer (label^*{...}, matching the Variable spelling; deserializer
grammar extension deferred -- conjugation currently arises only from
canonicalization at runtime, never from parsed input).
conjugate() toggles the marker and touches no slots; adjoint() is
deliberately unchanged (for BraKetSymmetry::Conjugate the swap IS the
adjoint, the conj being carried by the symmetry relation, so the marker
commutes through it -- certified by the new test).
This is the representation half of making the canonicalizer's
conjugation byproduct symbolic: instead of a network-level bool
(SlotCanonicalizationMetadata::conj / EvalExpr::canon_conj_), the
orientation fold will toggle conjugated_ on the tensor itself, giving
per-tensor granularity by construction.
fold_conjugate_braket now toggles the new elementwise-conjugation marker
(AbstractTensor::_conjugate(), implemented by Tensor) when it reorients
a BraKetSymmetry::Conjugate tensor: by the symmetry relation T{q;p} =
conj(T{p;q}) the swapped spelling denotes the conjugate value, so the
in-place toggle keeps the represented value invariant and the byproduct
becomes part of the expression instead of a side-channel bool. The
legacy bool return stays until every consumer reads the marker.
EvalExpr flat leaves acquire the leaf-hash invariant: the hash is always
that of the unstarred spelling, so the two orientations of a Conjugate
tensor keep sharing one cache slot while expr_ carries the symbolic
star; binarize's Adjoint wrapper serves the conjugation on retrieval.
…round-trip is now lossless
Drop the exploit_conjugate opt-ins: DefaultTensorCanonicalizer and TNv3 fold the two orientations of a BraKetSymmetry::Conjugate tensor onto one canonical spelling chosen by the content-based (presentation-invariant) prefer_swapped_braket rule, shared by the atom and network routes. Reserved antisymmetrizer/symmetrizer/transposition labels never reorient; identity (diagonal-trace) bundles never conjugate; c-number guards keep NormalOperators out of the fold. Eval-layer intermediates compute their bra/ket partition on the value orientation (value_oriented) so folded leaves do not merge distinct partitions. Tensor's Hermiticity-taking ctors preserve literal Conjugate for empty-bra+ket tensors.
Declarations state their true braket symmetry (NonHermitian where legacy code relied on the deserializer's Conjugate fallback); expected spellings re-blessed to the folded canonical forms; spintrace sections run under a cloned Field::Real registry (real_orbital_context); legacy TNV1/V2 cases hidden behind [.legacy-tn] per TNV3-only support policy.
The loop started at named_indices.size(), assuming the leading edges are the named ones. A named index that is not an edge (e.g. a pure proto index) shifted that cutoff onto an anonymous edge; the skipped edge's ordinal was then handed to another edge of the same space, yielding a non-injective rewrite that duplicated a slot index. Latent until the Conjugate braket fold reordered the edge sort.
A marker-conjugated (folded) tensor spells conj(bra<->ket-swapped); csv_transform / density_fit / tensor_hypercontract rebuilt tensors from the raw slot layout and silently dropped the conjugation, producing value-wrong factorizations of folded inputs. Normalize each rule's input tensor to the value orientation at entry; value_oriented moves from eval_expr.cpp's file scope to core/expressions/tensor.hpp.
expand_antisymm's raw-permutation rebuild and swap_spin's flavor relabeling are slot-preserving but reconstructed tensors from parts, silently dropping Tensor::conjugated(): a folded g^* expanded into raw NonSymm leaves lost its conjugation (value-wrong Kramers-traced energy in downstream consumers). Copy the marker on every slot-preserving rebuild.
Delete TensorBlockCanonicalizer::fold_conjugate_braket and orient_braket_by_color (zero callers; apply_canonical_braket_orientation is the live fold) and repoint the comments that named them. Document SlotCanonicalizationMetadata::conj as the swap parity with its single-consumer invariant, and give its detection the same c-number guard as create_graph. Assert the real-field no-marker precondition at the three spin.cpp rebuild sites that drop the conjugation marker (swap_bra_ket, remove_spin, merge_tensors). Retire stale exploit_conjugate test tags and fix comment typos.
First-ever compile of these TUs (TA-enabled test build now works against mpqc4's release _deps): the deserializer's Conjugate fallback let the always-on braket fold treat unrelated random test tensors as Hermitian partners (yielder key misses, changed tree shapes). Declare the data's true symmetry, matching the symbolic-suite policy. The cache_manager_batch_axis_veto keep-predicate section still fails with honest declarations — batching-era logic, never compiled before, likely pre-existing.
…ies the marker The strict braket sanity check on dummy edges predates the canonical braket-orientation fold: it only knew BraKetSymmetry::Symm as orientation-free, so a foldable Conjugate tensor spelled in the swapped orientation made a legal bra-bra/ket-ket contraction edge trip the assert (every Debug/assert-enabled CI job died there). The check now uses the fold's own predicate: Symm, or Conjugate and c-number and not pinned. Also remove_spin now carries the elementwise-conjugation marker through its relabeling rebuild (relabeling commutes with conjugation) -- the naive-V1 spintrace path reaches it with folded tensors, which the new no-marker assert caught.
Declare orientation-rigid tensors (amplitudes, DF factors, residual heads) braket-Nonsymm in the eval/btas/tapp/extract_subtrees tests and in the cost_analysis / external-interface example inputs, so the canonical braket-orientation fold does not reorient them: tree-shape predicates, yielder cache keys, and the cost_analysis reference outputs all stay put (cost_analysis references are unchanged). Port the ToT adjoint end-to-end TA test. The ITF references are regenerated: the canonicalization rework relabels CSE intermediates (value-preserving, a few more intermediates than before).
create_graph no longer rejects a dummy connecting bra-to-bra through an adjoint braket-Conjugate c-number tensor: the orientation fold may spell such a tensor bra<->ket swapped, so the connection is legal. Pin that with REQUIRE_NOTHROW and keep the covariance check exercised through a braket-Nonsymm pair, which cannot be reoriented and must still throw.
A
BraKetSymmetry::Conjugatetensor satisfiesT{p;q} = conj(T{q;p}), but the two spellings were treated as distinct: equivalent expressions canonicalized apart, and the evaluator computed and cached both orientations.This PR makes canonicalization pick one orientation per tensor.
Tensorgains a conjugation flag (conjugate()/conjugated(), renderedT^*= the elementwise complex conjugate ofT); when canonicalization spells a Conjugate tensor in the swapped orientation it sets the flag, so the represented value never changes. The flag participates in hashing/equality, round-trips through the serializer (T^*{...}), and is preserved by the mbpt transforms (csv/df/thc rules,expand_antisymm,swap_spin).prefer_swapped_braket), so the canonical spelling is independent of input order. (The earlier commits added an opt-in network-level bool; per the review that design was replaced, and the opt-in channel is gone — this description is of the final state.)NormalOperatorswap exchanges creators/annihilators), the reserved (anti)symmetrizers, and identity swaps (T{p,q;p,q}).binarizeserves a flagged leaf throughEvalOp::Adjoint.canonicalize_slotsreports the swap inmetadata.conj, consumed only by the single-tensor leaf constructor.Also in this PR:
Edge::add_vertexthrow). Now skipped per edge; regression test included.fold_conjugate_pairs_of_real_sum: folds{s, adjoint(s)}summand pairs of a real-valued sum into2*s(used downstream by MPQC's Kramers-restricted CC).default_idxptr_slottype_lesscompare{}explicitly instead of{}(an emptystd::functionsilently selected the space-only fallback comparator).Behavior changes to note: since the fold is unconditional, non-Hermitian tensors (e.g. amplitudes) must be declared
Nonsymm— the test suite was swept accordingly;Tensor::canonicalize()now canonicalizes even when no canonicalizer was registered (DefaultTensorCanonicalizerseeded underL""); the shared TN test runs TNV3-only (V1/V2 cannot ingest folded spellings, hidden as[.legacy-tn]).