From 25fd9ec9cc11938896f249ed4612afd7124ede49 Mon Sep 17 00:00:00 2001 From: Karl Kauc Date: Sat, 29 Aug 2026 19:18:57 +0200 Subject: [PATCH] fix(schematron): un-shadow Option/Future underlying rules, add underlyings to sample ISO Schematron fires only the first matching rule per pattern. In asset-validations the derivative-exposure warning rule (OP or FU or FX or SW) claimed every Option and Future, so the two "must have underlying" ERROR rules were dead code (SVRL: svrl:suppressed-rule). Same bug class as the earlier percentage-sum fix. Move them into their own pattern asset-underlying-validations (ruleset now has 11 patterns). The canonical sample had four options/futures without Underlyings, so un-shadowing would have turned "0 errors" into 4. Add an XSD-valid Underlyings/Underlying (Type, Name, Identifiers) to each: Allianz SE and Volkswagen AG (Security/ISIN), EURO STOXX 50 (Index/ISIN), EUR/USD (Currency/OtherID). Regenerate both negative fixtures from the new sample with the sed recipes in tests/fixtures/invalid/README.md; the Schematron fixture still fails with exactly 1 error (percentage sum). Align the Basic XSLT share-class price check with the Schematron bands (OK < 0.01, ROUNDING 0.01-0.1, ERROR >= 0.1); it previously only flagged ERROR from 1.0, contradicting the documented "matching tolerances". Verified: sample XSD-valid; Java Schematron 0 errors / 12 warnings on the sample, 1 error on the fixture, 1 error when an Underlyings block is removed; XSLT DQ report 2x OK; CSV 21 rows, TPT 42 rows, top-holdings 5, data binding 21 positions, DB import/export XSD-valid, Santuario sign/verify/tamper roundtrip unchanged. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YXZWhwE5EbHSybt55K6oUZ --- .../4.2.9/positions/Mixed-Fund_Positions.xml | 36 +++++++++++++++++++ .../Basic_Checks/README.md | 18 +++++++++- .../Basic_Checks/basic_checks.sch | 12 +++++++ .../Basic_Checks/README.md | 10 ++++-- .../Basic_Checks/basic_checks.xslt | 7 ++-- .../invalid/schematron-invalid_Positions.xml | 36 +++++++++++++++++++ .../invalid/xsd-invalid_Positions.xml | 36 +++++++++++++++++++ 7 files changed, 149 insertions(+), 6 deletions(-) diff --git a/FundsXML_Files/4.2.9/positions/Mixed-Fund_Positions.xml b/FundsXML_Files/4.2.9/positions/Mixed-Fund_Positions.xml index e81d234..0231a3d 100644 --- a/FundsXML_Files/4.2.9/positions/Mixed-Fund_Positions.xml +++ b/FundsXML_Files/4.2.9/positions/Mixed-Fund_Positions.xml @@ -639,6 +639,15 @@ 280 + + + Security + Allianz SE + + DE0008404005 + + + @@ -661,6 +670,15 @@ 85 + + + Security + Volkswagen AG (Vz) + + DE0007664039 + + + @@ -679,6 +697,15 @@ 10 2025-12-19 XEUR + + + Index + EURO STOXX 50 Index + + EU0009658145 + + + @@ -696,6 +723,15 @@ 125000 2026-03-16 XCME + + + Currency + EUR/USD + + EURUSD + + + diff --git a/Schematron_DataQuality_Checks/Basic_Checks/README.md b/Schematron_DataQuality_Checks/Basic_Checks/README.md index 7b4cba0..4478f4a 100644 --- a/Schematron_DataQuality_Checks/Basic_Checks/README.md +++ b/Schematron_DataQuality_Checks/Basic_Checks/README.md @@ -8,7 +8,7 @@ This directory contains comprehensive Schematron validation rules for FundsXML d |----------|-------| | **File** | `basic_checks.sch` | | **Query Binding** | XSLT 2.0 (`queryBinding="xslt2"`) | -| **Patterns** | 9 validation patterns | +| **Patterns** | 11 validation patterns | | **Total Rules** | 40+ assertions and reports | | **Purpose** | Comprehensive FundsXML data quality validation | @@ -25,6 +25,14 @@ direction rule). These were split into their own patterns — document with percentages summing to 120 % now produces an ERROR; the canonical sample (summing to 100 %) does not. +The same bug hid in `asset-validations`: the *Derivative Exposure* warning +rule (`OP or FU or FX or SW`) claimed every Option and Future first, so the +*Option Underlying* / *Future Underlying* ERROR rules never fired (SVRL listed +them as `svrl:suppressed-rule`). They now live in their own pattern, +`asset-underlying-validations`, and the canonical sample carries an +`Underlyings/Underlying` on each of its four options/futures so it still +passes with 0 errors. + Run it across stacks via [`invocation/`](invocation/) (CLI, native Java, Python/saxonche, .NET). The canonical sample yields **0 errors + 12 advisory warnings** (the broad `ShareClass` rule also matches `AssetDetails/ShareClass`; @@ -84,6 +92,14 @@ Asset type-specific requirements based on the AssetType code. | ISIN Required | EQ, BO, SC | ERROR | Equity, Bond, ShareClass assets must have ISIN | | Counterparty ID | AC | WARNING | Account assets should have counterparty LEI or BIC | | Derivative Exposure | OP, FU, FX, SW | WARNING | Derivatives should have exposure information | + +### Pattern 4b: Derivative Underlyings (`asset-underlying-validations`) + +Own pattern so these rules are not shadowed by the Derivative Exposure rule +above (see *Known ruleset fix*). + +| Rule | Asset Types | Severity | Description | +|------|-------------|----------|-------------| | Option Underlying | OP | ERROR | Options must have at least one underlying | | Future Underlying | FU | ERROR | Futures must have at least one underlying | diff --git a/Schematron_DataQuality_Checks/Basic_Checks/basic_checks.sch b/Schematron_DataQuality_Checks/Basic_Checks/basic_checks.sch index 3d70fed..fea0ad7 100644 --- a/Schematron_DataQuality_Checks/Basic_Checks/basic_checks.sch +++ b/Schematron_DataQuality_Checks/Basic_Checks/basic_checks.sch @@ -216,6 +216,18 @@ + + + + diff --git a/XSLT_DataQuality_Checks/Basic_Checks/README.md b/XSLT_DataQuality_Checks/Basic_Checks/README.md index 6860952..ec237a3 100644 --- a/XSLT_DataQuality_Checks/Basic_Checks/README.md +++ b/XSLT_DataQuality_Checks/Basic_Checks/README.md @@ -64,9 +64,13 @@ Compares calculated price to reported price: | Result | Difference | Status | |--------|------------|--------| -| OK | < 0.1 | ✓ OK | -| ROUNDING | 0.1 - 1.0 | ⚠ ROUNDING | -| ERROR | ≥ 1.0 | ✗ ERROR | +| OK | < 0.01 | ✓ OK | +| ROUNDING | 0.01 - 0.1 | ⚠ ROUNDING | +| ERROR | ≥ 0.1 | ✗ ERROR | + +These bands are identical to the Schematron `Price Calculation` / `Price +Rounding` rules in `Schematron_DataQuality_Checks/Basic_Checks/basic_checks.sch`; +change both together. ### 4. Portfolio Position Reconciliation diff --git a/XSLT_DataQuality_Checks/Basic_Checks/basic_checks.xslt b/XSLT_DataQuality_Checks/Basic_Checks/basic_checks.xslt index c3b4572..5eda711 100644 --- a/XSLT_DataQuality_Checks/Basic_Checks/basic_checks.xslt +++ b/XSLT_DataQuality_Checks/Basic_Checks/basic_checks.xslt @@ -242,10 +242,13 @@ - + + ✓ OK - + ⚠ ROUNDING diff --git a/tests/fixtures/invalid/schematron-invalid_Positions.xml b/tests/fixtures/invalid/schematron-invalid_Positions.xml index 980b455..18380e5 100644 --- a/tests/fixtures/invalid/schematron-invalid_Positions.xml +++ b/tests/fixtures/invalid/schematron-invalid_Positions.xml @@ -639,6 +639,15 @@ 280 + + + Security + Allianz SE + + DE0008404005 + + + @@ -661,6 +670,15 @@ 85 + + + Security + Volkswagen AG (Vz) + + DE0007664039 + + + @@ -679,6 +697,15 @@ 10 2025-12-19 XEUR + + + Index + EURO STOXX 50 Index + + EU0009658145 + + + @@ -696,6 +723,15 @@ 125000 2026-03-16 XCME + + + Currency + EUR/USD + + EURUSD + + + diff --git a/tests/fixtures/invalid/xsd-invalid_Positions.xml b/tests/fixtures/invalid/xsd-invalid_Positions.xml index 0d42e6b..ac4b68c 100644 --- a/tests/fixtures/invalid/xsd-invalid_Positions.xml +++ b/tests/fixtures/invalid/xsd-invalid_Positions.xml @@ -639,6 +639,15 @@ 280 + + + Security + Allianz SE + + DE0008404005 + + + @@ -661,6 +670,15 @@ 85 + + + Security + Volkswagen AG (Vz) + + DE0007664039 + + + @@ -679,6 +697,15 @@ 10 2025-12-19 XEUR + + + Index + EURO STOXX 50 Index + + EU0009658145 + + + @@ -696,6 +723,15 @@ 125000 2026-03-16 XCME + + + Currency + EUR/USD + + EURUSD + + +