Skip to content

Index an entity's four audit members, and expose them to lint rules - #1132

Merged
ako merged 2 commits into
mainfrom
fix/catalog-audit-members
Sep 17, 2026
Merged

ako merged 2 commits into
mainfrom
fix/catalog-audit-members

Conversation

@ako

@ako ako commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

Found while writing the MDL equivalent of an "add CreatedDate/ChangedDate to every entity" script: the query anyone would write returns a wrong answer, not an error.

SELECT e.QualifiedName FROM CATALOG.ENTITIES e
  LEFT JOIN CATALOG.ATTRIBUTES a
    ON a.EntityQualifiedName = e.QualifiedName AND a.Name = 'CreatedDate'
 WHERE a.Name IS NULL

reports every entity as missing CreatedDate forever, including ones that already have it. Measured before the change: adding the fields and running refresh catalog full left the result unchanged, and CATALOG.ATTRIBUTES for that entity still listed only its four ordinary attributes.

Why

The catalog wasn't dropping anything. Mendix stores Owner, ChangedBy, CreatedDate and ChangedDate as booleans on the entity's generalization node (NoGeneralization.HasCreatedDate), not as attributes — so they're absent from CATALOG.ATTRIBUTES by construction.

What makes it a trap is that DESCRIBE ENTITY renders them in the attribute list as CreatedDate: AutoCreatedDate. The one view a user checks against says they're attributes; the catalog says they don't exist.

Commit 1 — four columns on CATALOG.ENTITIES

SELECT QualifiedName FROM CATALOG.ENTITIES WHERE HasCreatedDate = 0;

Simpler than the join, and correct. Fabricating attribute rows was the alternative and is worse — it invents structure the model doesn't have and would disagree with AttributeCount.

Schema version 11 → 12. CREATE TABLE IF NOT EXISTS does not add a column to a cached catalog, so without the bump every query naming one fails with no such column on exactly the machines that already have a catalog.

Commit 2 — the same four on the Starlark entity

With the data indexed a rule still couldn't read it: entity carried has_event_handlers and is_external and nothing about the audit trail, and neither workaround exists — the members aren't attributes, so attributes_for() never yields them and attribute_count excludes them. Now has_created_date, has_changed_date, has_owner, has_changed_by.

One subtlety worth flagging for review: they're scanned as sql.NullInt64, not int. A row without the columns scans as NULL, NULL into a plain int fails the whole query, and Entities() swallows a query error by yielding nothing — so every rule sees zero entities and reports zero violations. A green run that checked nothing. Three rule tests went green with the entity table empty before I caught it.

The linter's fixtures declare their own copy of the entities schema and three INSERTed positionally, so a new column broke them; they now name their columns. That duplication is still a maintenance risk, worth collapsing onto the real schema separately.

Verification

Catalog, on a real 11.14 project:

before after
AuthToken HasCreatedDate/HasChangedDate 0/0, then 1/1 after add attribute if not exists CreatedDate: AutoCreatedDate
other four entities stay 0
WHERE HasCreatedDate = 0 5 rows (wrong) 4 rows
DESCRIBE CATALOG.ENTITIES lists all four

Lint, end to end with a QUAL900-style rule over the same project: names 6 entities, then 5 after adding the audit trail to one — and stops naming that one.

Controls. Stubbing the catalog values to literal zeros fails TestEntitiesCarryTheAuditMembers with the pre-fix symptom. Wiring the Starlark fields to starlark.Bool(false) makes one test report both entities instead of one and the other report none — so a field that silently reads False everywhere cannot pass.

make lint-go and make test pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BNDe35kDNsMX5cz4Ahn4rk

"Which entities have no audit trail?" was unanswerable from SQL, and the
query anyone would write returned a wrong answer rather than an error:

  SELECT e.QualifiedName FROM CATALOG.ENTITIES e
    LEFT JOIN CATALOG.ATTRIBUTES a
      ON a.EntityQualifiedName = e.QualifiedName AND a.Name = 'CreatedDate'
   WHERE a.Name IS NULL

reports every entity as missing CreatedDate, forever, including ones that
have it. Measured before this change: adding CreatedDate/ChangedDate to an
entity and rebuilding the catalog with `refresh catalog full` left the row
count unchanged, and CATALOG.ATTRIBUTES for that entity still listed only
its four ordinary attributes.

The catalog was not dropping anything. Mendix stores Owner, ChangedBy,
CreatedDate and ChangedDate as BOOLEANS on the entity's generalization node
(NoGeneralization.HasCreatedDate and friends), not as attributes, so they
are absent from CATALOG.ATTRIBUTES by construction and the builder was a
faithful projection of a model that has no such rows.

What made it a trap is that DESCRIBE ENTITY renders them IN the attribute
list, as `CreatedDate: AutoCreatedDate` — so the one view a user checks
against says they are attributes, and the catalog says they do not exist.

The fix follows the model rather than the rendering: four columns on
CATALOG.ENTITIES, mirroring the booleans that are actually stored, so

  SELECT QualifiedName FROM CATALOG.ENTITIES WHERE HasCreatedDate = 0

is both simpler than the join and correct. Fabricating attribute rows was
the alternative and is worse: it would invent structure the model does not
have, and disagree with AttributeCount, which counts real attributes.

Schema version 11 -> 12. CREATE TABLE IF NOT EXISTS does not add a column
to a cached catalog, so without the bump every query naming one fails with
"no such column" on exactly the machines that already have a catalog.

Verified end to end on a real 11.14 project: AuthToken reads 0/0 before and
1/1 after `add attribute if not exists CreatedDate: AutoCreatedDate`, the
other four entities stay 0, and the WHERE HasCreatedDate = 0 query goes
from 5 rows to 4. Stubbing the four values back to literal zeros fails
TestEntitiesCarryTheAuditMembers with the pre-fix symptom.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BNDe35kDNsMX5cz4Ahn4rk
@github-actions

Copy link
Copy Markdown

AI Code Review

Review Summary

This PR fixes a critical bug in the catalog where queries for entities missing audit fields (CreatedDate/ChangedDate/Owner/ChangedBy) returned incorrect results. The issue stemmed from Mendix storing these audit fields as booleans on the entity's generalization node (not as attributes), making them invisible to the original LEFT JOIN query against CATALOG.ATTRIBUTES.

Critical Issues

None found.

Moderate Issues

None found.

Minor Issues

None found.

What Looks Good

  • Correct root cause analysis: Properly identified that audit members are stored as booleans on generalization nodes, not as attributes
  • Appropriate fix: Added four INTEGER columns to CATALOG.ENTITIES mirroring the actual storage mechanism
  • Schema version bump: Correctly increased from 11 to 12 since CREATE TABLE IF NOT EXISTS doesn't add columns to existing tables
  • Comprehensive testing:
    • New test file builder_audit_members_test.go thoroughly verifies both positive and negative cases
    • Includes control test that validates the pre-fix symptom when values are stubbed to zero
    • Tests verify the exact query mentioned in the issue now works correctly
  • Clean implementation:
    • Uses boolToInt() helper for proper boolean-to-integer conversion
    • Maintains existing code patterns in the catalog builder
    • Includes detailed comments explaining why the issue was confusing (DESCRIBE ENTITY shows them as attributes)
  • Atomic scope: Focused exclusively on fixing the catalog audit members query without unrelated changes

Recommendation

Approve - This PR correctly fixes the bug with minimal, focused changes, includes comprehensive tests, and follows all project conventions. The solution accurately reflects how Mendix actually stores audit data and enables correct catalog queries for this important use case.


Automated review via OpenRouter (Nemotron Super 120B) — workflow source

Follow-up to the catalog columns in the previous commit: with the data
indexed, a rule still could not read it. `entity` carried
has_event_handlers and is_external and nothing about the audit trail, and
the two obvious workarounds do not exist -- the members are not attributes,
so attributes_for() never yields them and attribute_count excludes them.
"Which persistent entities have no CreatedDate" was unwritable as a rule.

Four fields: has_created_date, has_changed_date, has_owner, has_changed_by.

Scanned as sql.NullInt64 rather than int, which is load-bearing and is what
the first cut got wrong. A row without the columns -- any fixture that does
not name them -- scans as NULL, NULL into a plain int fails the whole query,
and Entities() swallows a query error by yielding nothing. Every rule then
sees zero entities and reports zero violations: a green run that checked
nothing. That is the same failure shape the catalog-mode bump guards
against, reached through the scan instead. Three rule tests went green with
the entity table empty before this was fixed.

The linter's test fixtures declare their own copy of the entities schema,
so adding a column broke them -- three of them INSERTed positionally. They
now name their columns, which is why a fourth column will not break them
again. That duplication is still a maintenance risk and is worth collapsing
onto the real schema separately.

Two tests, and the control is what makes them worth having: wiring all four
to starlark.Bool(false) makes the first report both entities instead of one
and the second report none, so a field that silently reads False everywhere
cannot pass. Verified end to end as well -- a QUAL900-style rule over a real
11.14 project names 6 entities, and after
`add attribute if not exists CreatedDate: AutoCreatedDate` on one of them it
names 5 and stops naming that one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BNDe35kDNsMX5cz4Ahn4rk
@ako ako changed the title Index an entity's four audit members so the catalog can answer for them Index an entity's four audit members, and expose them to lint rules Sep 17, 2026
@ako
ako merged commit 772069f into main Sep 17, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant