[WIP] sql schema and more

.claude/commands/sync-schema-guides.md

@@ -0,0 +1,100 @@
+---
+description: Evaluate whether a change touches the SQL-schema documentation set (SQLSCHEMA.md + both SQLSCHEMA-GUIDE docs + glossaries) and bring every affected document back in sync. Invoke after ANY change to the SQL schema artifacts, the SQL code generator, the PIM DTOs/serializers, or the Extend derived-property layer — or run it standalone as a consistency audit.
+argument-hint: [optional: short description of what changed, or "audit" for a full consistency check]
+---
+
+# /sync-schema-guides
+
+Keep the SQL-schema documentation set consistent with the code and with itself. The set:
+
+| Document | Role |
+|---|---|
+| `SysML2.NET.CodeGenerator/SQLSCHEMA.md` | Compact reference: artifact table, audit findings, measured numbers, verification recipe, guide section map |
+| `SysML2.NET.CodeGenerator/SQLSCHEMA-GUIDE.md` | Long-form architectural guide (English) — 19 sections + glossary appendix (§19) |
+| `SysML2.NET.CodeGenerator/SQLSCHEMA-GUIDE.nl.md` | Dutch translation of the guide — MUST stay structurally identical (same sections, same anchors pattern, same tables) |
+| `SysML2.NET.CodeGenerator/IMPACT-RADIUS.md` | Design sketch for the impact-radius engine (guide obligation §15.1). English-only. Update when propagation kinds, the `derived_dependency` catalog concept, closure indexes, or the §15/§18 obligations it cites change. |
+| **Schema Atlas artifact** (`https://claude.ai/code/artifact/02fcfe37-e5b2-473d-9716-679b2ac34b26`) | Published interactive visualization of the generated schema — all tables/columns/FKs/indexes/triggers, parsed live from `schema2.generated.sql`. Regeneration assets and refresh procedure: `.claude/schema-atlas/` (see its README.md). |
+
+**Language policy for the Dutch guide:** Dutch prose, English conceptual terms (derived
+properties, stored state, fold, checkpoint, overlay, impact radius, census, …). Never
+translate the conceptual vocabulary — it must match the English guide, the schema comments,
+and the code.
+
+## 1. Determine what changed (skip if invoked with an explicit change description)
+
+```
+git status --porcelain
+git diff --stat HEAD
+```
+
+The documentation set is AFFECTED when a change touches any of:
+
+- `SysML2.NET.CodeGenerator/Sql/**` (golden schema, generated schema, smoke test)
+- `SysML2.NET.CodeGenerator/Templates/Uml/core-sql-schema-2.hbs`
+- `SysML2.NET.CodeGenerator/HandleBarHelpers/SqlSchemaHelpers.cs`
+- `SysML2.NET.CodeGenerator/Extensions/SqlSchemaExtensions.cs`
+- `SysML2.NET.CodeGenerator/Generators/UmlHandleBarsGenerators/SQLSchemaGenerator.cs`
+- `SysML2.NET.CodeGenerator/Generators/UmlHandleBarsGenerators/ClassKindRegistry.cs` (and the
+  `ClassKindRegistration` / `ModelVersionRegistration` records) — the append-only registry
+  freezing class_kind ids and model_version ordinals; any append/close here changes the
+  multi-version narrative (guide §6.4/§12.1) and the seed counts
+- `SysML2.NET/PIM/**` and `SysML2.NET.Serializer.Json/PIM/**` (the PIM the schema §3 models)
+- `SysML2.NET/Extend/**` (only when it changes WHICH derived properties exist or their
+  semantics — the guides' census numbers and impact-radius narrative depend on them)
+- `Resources/KerML_only_xmi.uml` / `Resources/SysML_only_xmi.uml` (metamodel change ⇒ census
+  numbers, table counts, catalog counts all suspect)
+
+If nothing relevant changed and the argument is not "audit": report "no sync needed" and stop.
+
+## 2. Sync procedure (English guide is the master)
+
+1. **Update the English guide first**, then mirror into the Dutch guide, then update
+   SQLSCHEMA.md's compact statements and its guide section map. Never let the three diverge.
+2. **Numbers are load-bearing — verify, don't trust.** These recur across all three documents
+   and MUST match reality after any schema/generator change:
+   - smoke-test assertion count (`grep -c "RAISE NOTICE 'PASS" SysML2.NET.CodeGenerator/Sql/schema.smoke.sql`)
+   - subtype-table count (currently 47), link-table count (7), enum count (7), view count (167),
+     class_kind rows (175, ids frozen by ClassKindRegistry), model_version rows (currently 1),
+     reference-validation sources in the two-tier validate functions (currently 42),
+     partitioned-table count × modulus
+   - measured performance numbers: only replace with NEW measurements, never extrapolate
+     silently — label extrapolations as such
+3. **Section integrity:** the guides' section numbers are referenced from SQLSCHEMA.md's
+   section map and from within the guides themselves. When adding a section, prefer appending
+   subsections (x.y) over renumbering; if renumbering is unavoidable, grep all three documents
+   for stale `section N` / `§N` / anchor references and fix the TOCs.
+4. **Glossary appendix (§19 in both guides):** for every new term of art introduced by the
+   change, add a row — shortest clear definition + guide-section reference — to BOTH
+   glossaries, alphabetically placed. When a section is renumbered, re-verify every reference
+   in the *See*/*Zie* columns.
+5. **Schema-file § banners vs guide sections:** the schema files' `§N` banners are a separate
+   numbering space. If a banner is added/renumbered in `schema.golden.sql`, update the
+   template in lockstep (hand-written sections must stay byte-identical) and fix the `(§N)`
+   suffixes in the guides' section headings.
+6. **Republish the Schema Atlas artifact** whenever `Sql/schema2.generated.sql` changed
+   (its content is parsed from that file):
+   - run `.claude/schema-atlas/build-schema-atlas.ps1` (writes `%TEMP%\schema-atlas.html`
+     and prints the freshly parsed counts);
+   - if any printed count differs from the literals hard-coded in
+     `.claude/schema-atlas/template.html` (masthead lede, layer blurbs, SVG annotations:
+     175 class kinds, 47 subtype / 7 link tables, 58 × 16 = 928 partitions, 2,629 cloned
+     FKs, 167 views, 3 triggers), fix the template first and rebuild;
+   - publish the output file with the Artifact tool, passing
+     `url: https://claude.ai/code/artifact/02fcfe37-e5b2-473d-9716-679b2ac34b26` so the
+     existing artifact updates in place (favicon stays `🐘`).
+
+## 3. Verification before reporting done
+
+- The three documents agree on every shared number and section reference.
+- If `schema2.generated.sql` changed: the Schema Atlas artifact was republished (step 6) and
+  its stats strip matches the build script's printed counts.
+- Both guides have identical section structure (`grep -c "^## " both files` — counts match).
+- If the schema itself changed: the generator fixture passes and the smoke test passes
+  against both golden and generated schema (see SQLSCHEMA.md → Verification).
+- Report a short diff summary per document: what was updated and why.
+
+## Standing rule (also enforced via CLAUDE.md)
+
+Any task that edits the affected paths of step 1 is NOT complete until this evaluation has
+run. A change that alters behavior, counts, measured numbers, invariants, or terminology and
+leaves the documentation set untouched is a defect, not a shortcut.

.claude/schema-atlas/README.md

@@ -0,0 +1,39 @@
+# SysML2 Schema Atlas
+
+An interactive visualization of the generated PostgreSQL persistence schema, published as a
+Claude Code Artifact:
+
+**https://claude.ai/code/artifact/02fcfe37-e5b2-473d-9716-679b2ac34b26**
+
+The page shows all tables with every column, foreign key, CHECK constraint, index, and
+trigger; an architecture diagram of the reference mechanism; a FK reference map grouped by
+target; and the enum/function/view appendices. All table-level data is parsed directly from
+`SysML2.NET.CodeGenerator/Sql/schema2.generated.sql`, so the page is only as fresh as its
+last publish.
+
+## Keeping it up to date (part of the `/sync-schema-guides` obligation)
+
+Whenever `SysML2.NET.CodeGenerator/Sql/schema2.generated.sql` changes, the artifact must be
+refreshed as part of the same task (the `/sync-schema-guides` skill carries this step):
+
+1. Build the page (parses the schema, splices data + snapshot stamp into the template):
+
+   ```powershell
+   .claude/schema-atlas/build-schema-atlas.ps1            # writes %TEMP%\schema-atlas.html
+   ```
+
+2. Publish the output file with the Artifact tool, passing the URL above as `url` so the
+   existing artifact is updated in place (favicon stays `🐘`).
+
+3. Hard-coded prose numbers: the stats strip and catalog are fully data-driven, but the
+   masthead lede, the layer blurbs, and the SVG diagram annotations carry a few literals
+   (175 class kinds, 47 subtype / 7 link tables, 58 × 16 = 928 partitions, 2,629 cloned FK
+   constraints, 167 views, 3 triggers). The build script prints the freshly parsed counts —
+   if any differ from those literals, update `template.html` accordingly before publishing.
+
+## Files
+
+| File | Role |
+|---|---|
+| `build-schema-atlas.ps1` | Parses `schema2.generated.sql` → JSON, splices into the template, writes the final page to `%TEMP%` |
+| `template.html` | The page (styles, diagram, rendering JS) with `__SCHEMA_JSON__` / `__SNAPSHOT__` placeholders |

.claude/schema-atlas/build-schema-atlas.ps1

@@ -0,0 +1,205 @@
+# Builds the "SysML2 Schema Atlas" artifact page from the generated SQL schema.
+#
+# Parses SysML2.NET.CodeGenerator/Sql/schema2.generated.sql into JSON (tables, columns,
+# FKs, checks, indexes, triggers, functions, views, enums, partition/seed stats), splices
+# it into .claude/schema-atlas/template.html, and writes the finished page to the output
+# path (default: %TEMP%\schema-atlas.html). The page is then published as an Artifact —
+# see .claude/schema-atlas/README.md for the artifact URL and publish step.
+#
+# Windows PowerShell 5.1 compatible. No repo files are modified.
+
+param(
+    [string]$OutFile = (Join-Path $env:TEMP 'schema-atlas.html')
+)
+
+$ErrorActionPreference = 'Stop'
+
+$repoRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
+$src = Join-Path $repoRoot 'SysML2.NET.CodeGenerator\Sql\schema2.generated.sql'
+$templatePath = Join-Path $PSScriptRoot 'template.html'
+
+$lines = Get-Content $src -Encoding UTF8
+
+# --- section banners ---
+$sections = @()
+for ($i = 0; $i -lt $lines.Count; $i++) {
+    if ($lines[$i] -match '^-- (\d+)\. ([A-Z].*?)\s*(\[GENERATED\]|\[HAND-WRITTEN\])?\s*$') {
+        $sections += [pscustomobject]@{ num = [int]$Matches[1]; title = $Matches[2].Trim(); origin = $(if ($Matches[3]) { $Matches[3].Trim('[',']') } else { '' }); line = $i }
+    }
+}
+
+function Get-Section($lineNo) {
+    $best = $null
+    foreach ($s in $sections) { if ($s.line -le $lineNo) { $best = $s } }
+    return $best
+}
+
+# --- enums ---
+$enums = @()
+foreach ($l in $lines) {
+    if ($l -match "^CREATE TYPE sysml2\.(\w+) AS ENUM \((.+)\);") {
+        $vals = ($Matches[2] -split ',\s*') | ForEach-Object { $_.Trim(" '") }
+        $enums += [pscustomobject]@{ name = $Matches[1]; values = $vals }
+    }
+}
+
+# --- tables ---
+$tables = @()
+for ($i = 0; $i -lt $lines.Count; $i++) {
+    if ($lines[$i] -notmatch '^CREATE TABLE sysml2\.(\w+) \($') { continue }
+    $tname = $Matches[1]
+    $startLine = $i
+
+    # preceding contiguous comment block
+    $cmt = @()
+    $j = $i - 1
+    while ($j -ge 0 -and $lines[$j] -match '^--( |$)') { $cmt = ,($lines[$j] -replace '^--\s?', '') + $cmt; $j-- }
+
+    $cols = @(); $pk = ''; $fks = @(); $uniques = @(); $checks = @(); $partition = ''
+    $pendingCmt = @()
+    $i++
+    while ($i -lt $lines.Count) {
+        $l = $lines[$i]
+        if ($l -match '^\)\s*(PARTITION BY .+?)?;') { if ($Matches[1]) { $partition = $Matches[1] }; break }
+        $t = $l.Trim().TrimEnd(',')
+        if ($t -eq '') { $i++; continue }
+        if ($t -match '^--\s?(.*)$') { $pendingCmt += $Matches[1]; $i++; continue }
+
+        $inline = ''
+        if ($t -match '^(.*?)\s+--\s?(.*)$' -and $t -notmatch "'.*--.*'") { $t = $Matches[1].Trim(); $inline = $Matches[2] }
+
+        if ($t -match '^FOREIGN KEY \((.+?)\)$') {
+            # REFERENCES clause continues on the next line
+            $i++
+            $t = ($t + ' ' + $lines[$i].Trim().TrimEnd(','))
+        }
+
+        if ($t -match '^PRIMARY KEY \((.+)\)$') { $pk = $Matches[1] }
+        elseif ($t -match '^FOREIGN KEY \((.+?)\) REFERENCES sysml2\.(\w+) \((.+?)\)\s*(ON DELETE \w+( \w+)?)?$') {
+            $fks += [pscustomobject]@{ cols = $Matches[1]; refTable = $Matches[2]; refCols = $Matches[3]; onDelete = $(if ($Matches[4]) { $Matches[4] } else { '' }) }
+        }
+        elseif ($t -match '^UNIQUE \((.+)\)$') { $uniques += $Matches[1] }
+        elseif ($t -match '^CONSTRAINT (\w+)$') {
+            # multi-line named CHECK: accumulate following lines until parens balance
+            $chkName = $Matches[1]; $body = ''
+            do {
+                $i++
+                $body += ' ' + $lines[$i].Trim().TrimEnd(',')
+                $open = ($body.ToCharArray() | Where-Object { $_ -eq '(' }).Count
+                $close = ($body.ToCharArray() | Where-Object { $_ -eq ')' }).Count
+            } while ($open -ne $close -and $i -lt $lines.Count)
+            $checks += [pscustomobject]@{ name = $chkName; body = ($body.Trim() -replace '^CHECK\s*', '') }
+        }
+        elseif ($t -match '^(CONSTRAINT (\w+) )?CHECK\s*(.+)$') {
+            $checks += [pscustomobject]@{ name = $(if ($Matches[2]) { $Matches[2] } else { '' }); body = $Matches[3] }
+        }
+        elseif ($t -match '^(\w+)\s+([\w\.]+)\s*(.*)$') {
+            $cname = $Matches[1]; $ctype = $Matches[2]; $rest = $Matches[3]
+            $notnull = $rest -match 'NOT NULL'
+            $default = ''; if ($rest -match 'DEFAULT (.+?)( NOT NULL| NULL| REFERENCES|$)') { $default = $Matches[1].Trim() }
+            $ref = $null
+            if ($rest -match 'REFERENCES sysml2\.(\w+) \((\w+)\)\s*(ON DELETE \w+( \w+)?)?') {
+                $ref = [pscustomobject]@{ table = $Matches[1]; col = $Matches[2]; onDelete = $(if ($Matches[3]) { $Matches[3] } else { '' }) }
+            }
+            $cols += [pscustomobject]@{
+                name = $cname; type = $ctype; notnull = $notnull; default = $default; ref = $ref
+                comment = (($pendingCmt + $inline) | Where-Object { $_ }) -join ' '
+            }
+            $pendingCmt = @()
+        }
+        $i++
+    }
+    $sec = Get-Section $startLine
+    $tables += [pscustomobject]@{
+        name = $tname; section = $sec.num; comment = ($cmt -join "`n")
+        columns = $cols; pk = $pk; fks = $fks; uniques = $uniques; checks = $checks; partition = $partition
+    }
+}
+
+# --- indexes (statements may span lines; read until ';') ---
+$indexes = @()
+for ($i = 0; $i -lt $lines.Count; $i++) {
+    if ($lines[$i] -match '^CREATE (UNIQUE )?INDEX (\w+)') {
+        $uniq = [bool]$Matches[1]; $iname = $Matches[2]
+        $stmt = $lines[$i]
+        while ($stmt -notmatch ';\s*$') { $i++; $stmt += ' ' + $lines[$i].Trim() }
+        if ($stmt -match 'ON sysml2\.(\w+)\s*(.*);') {
+            $indexes += [pscustomobject]@{ name = $iname; unique = $uniq; table = $Matches[1]; def = $Matches[2].Trim() }
+        }
+    }
+}
+
+# --- triggers ---
+$triggers = @()
+for ($i = 0; $i -lt $lines.Count; $i++) {
+    if ($lines[$i] -match '^CREATE TRIGGER (\w+)$') {
+        $trg = $Matches[1]; $timing = ''; $tbl = ''; $fn = ''
+        for ($k = $i + 1; $k -lt $i + 5; $k++) {
+            if ($lines[$k] -match '(BEFORE|AFTER) (INSERT|UPDATE|DELETE)( OR \w+)* ON sysml2\.(\w+)') { $timing = $Matches[0] -replace ' ON sysml2\.\w+', ''; $tbl = $Matches[4] }
+            if ($lines[$k] -match 'EXECUTE FUNCTION sysml2\.(\w+)') { $fn = $Matches[1] }
+        }
+        $triggers += [pscustomobject]@{ name = $trg; table = $tbl; timing = $timing; fn = $fn }
+    }
+}
+
+# --- functions ---
+$functions = @()
+for ($i = 0; $i -lt $lines.Count; $i++) {
+    if ($lines[$i] -match '^CREATE OR REPLACE FUNCTION sysml2\.(\w+)\(') {
+        $fname = $Matches[1]
+        $ret = ''
+        for ($k = $i; $k -lt $i + 15; $k++) { if ($lines[$k] -match '^\s*RETURNS (.+?)\s*$') { $ret = $Matches[1]; break } }
+        $cmt = @(); $j = $i - 1
+        while ($j -ge 0 -and $lines[$j] -match '^--( |$)') { $cmt = ,($lines[$j] -replace '^--\s?', '') + $cmt; $j-- }
+        $functions += [pscustomobject]@{ name = $fname; returns = $ret; comment = (($cmt | Select-Object -First 4) -join ' ') }
+    }
+}
+
+# --- views ---
+$views = @()
+foreach ($l in $lines) { if ($l -match '^CREATE VIEW sysml2\.(\w+) AS$') { $views += $Matches[1] } }
+
+# --- partitions: modulus from the DO loop constant; partitioned tables carry a partition clause ---
+$partModulus = 0
+foreach ($l in $lines) { if ($l -match 'partition_count\s+constant int := (\d+)') { $partModulus = [int]$Matches[1]; break } }
+$partitionedTables = @($tables | Where-Object { $_.partition }).Count
+
+# --- class_kind seed rows ---
+$ckText = [System.IO.File]::ReadAllText($src)
+$ckBlock = [regex]::Match($ckText, "(?s)INSERT INTO sysml2\.class_kind.*?VALUES(.*?);")
+$classKindRows = ([regex]::Matches($ckBlock.Groups[1].Value, "\(\d+, '")).Count
+$abstractKinds = ([regex]::Matches($ckBlock.Groups[1].Value, "', true, ")).Count
+
+$result = [pscustomobject]@{
+    generatedAt = ''
+    sections = $sections | Select-Object num, title, origin
+    enums = $enums
+    tables = $tables
+    indexes = $indexes
+    triggers = $triggers
+    functions = $functions
+    views = $views
+    partitions = [pscustomobject]@{ modulus = $partModulus; partitionedTables = $partitionedTables }
+    classKindRows = $classKindRows
+    abstractKinds = $abstractKinds
+}
+
+$json = $result | ConvertTo-Json -Depth 8
+
+# --- splice into template ---
+$html = [System.IO.File]::ReadAllText($templatePath)
+
+if (-not $html.Contains('__SCHEMA_JSON__')) { throw 'template.html is missing the __SCHEMA_JSON__ placeholder' }
+
+$branch = (& git -C $repoRoot rev-parse --abbrev-ref HEAD 2>$null)
+
+if (-not $branch) { $branch = 'unknown' }
+
+$snapshot = "branch <code>$branch</code>, $(Get-Date -Format 'yyyy-MM-dd')"
+$html = $html.Replace('__SCHEMA_JSON__', $json.Replace('</', '<\/'))
+$html = $html.Replace('__SNAPSHOT__', $snapshot)
+
+[System.IO.File]::WriteAllText($OutFile, $html, (New-Object System.Text.UTF8Encoding($false)))
+
+Write-Host "tables=$($tables.Count) enums=$($enums.Count) indexes=$($indexes.Count) triggers=$($triggers.Count) functions=$($functions.Count) views=$($views.Count) partitionedTables=$partitionedTables modulus=$partModulus classKindRows=$classKindRows abstract=$abstractKinds"
+Write-Host "wrote $OutFile ($([math]::Round((Get-Item $OutFile).Length / 1KB)) KB)"