Index an entity's four audit members, and expose them to lint rules - #1132
Merged
Merged
Conversation
"Which entities have no audit trail?" was unanswerable from SQL, and the
query anyone would write returned a wrong answer rather than an error:
SELECT e.QualifiedName FROM CATALOG.ENTITIES e
LEFT JOIN CATALOG.ATTRIBUTES a
ON a.EntityQualifiedName = e.QualifiedName AND a.Name = 'CreatedDate'
WHERE a.Name IS NULL
reports every entity as missing CreatedDate, forever, including ones that
have it. Measured before this change: adding CreatedDate/ChangedDate to an
entity and rebuilding the catalog with `refresh catalog full` left the row
count unchanged, and CATALOG.ATTRIBUTES for that entity still listed only
its four ordinary attributes.
The catalog was not dropping anything. Mendix stores Owner, ChangedBy,
CreatedDate and ChangedDate as BOOLEANS on the entity's generalization node
(NoGeneralization.HasCreatedDate and friends), not as attributes, so they
are absent from CATALOG.ATTRIBUTES by construction and the builder was a
faithful projection of a model that has no such rows.
What made it a trap is that DESCRIBE ENTITY renders them IN the attribute
list, as `CreatedDate: AutoCreatedDate` — so the one view a user checks
against says they are attributes, and the catalog says they do not exist.
The fix follows the model rather than the rendering: four columns on
CATALOG.ENTITIES, mirroring the booleans that are actually stored, so
SELECT QualifiedName FROM CATALOG.ENTITIES WHERE HasCreatedDate = 0
is both simpler than the join and correct. Fabricating attribute rows was
the alternative and is worse: it would invent structure the model does not
have, and disagree with AttributeCount, which counts real attributes.
Schema version 11 -> 12. CREATE TABLE IF NOT EXISTS does not add a column
to a cached catalog, so without the bump every query naming one fails with
"no such column" on exactly the machines that already have a catalog.
Verified end to end on a real 11.14 project: AuthToken reads 0/0 before and
1/1 after `add attribute if not exists CreatedDate: AutoCreatedDate`, the
other four entities stay 0, and the WHERE HasCreatedDate = 0 query goes
from 5 rows to 4. Stubbing the four values back to literal zeros fails
TestEntitiesCarryTheAuditMembers with the pre-fix symptom.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BNDe35kDNsMX5cz4Ahn4rk
AI Code ReviewReview SummaryThis PR fixes a critical bug in the catalog where queries for entities missing audit fields (CreatedDate/ChangedDate/Owner/ChangedBy) returned incorrect results. The issue stemmed from Mendix storing these audit fields as booleans on the entity's generalization node (not as attributes), making them invisible to the original LEFT JOIN query against CATALOG.ATTRIBUTES. Critical IssuesNone found. Moderate IssuesNone found. Minor IssuesNone found. What Looks Good
RecommendationApprove - This PR correctly fixes the bug with minimal, focused changes, includes comprehensive tests, and follows all project conventions. The solution accurately reflects how Mendix actually stores audit data and enables correct catalog queries for this important use case. Automated review via OpenRouter (Nemotron Super 120B) — workflow source |
Follow-up to the catalog columns in the previous commit: with the data indexed, a rule still could not read it. `entity` carried has_event_handlers and is_external and nothing about the audit trail, and the two obvious workarounds do not exist -- the members are not attributes, so attributes_for() never yields them and attribute_count excludes them. "Which persistent entities have no CreatedDate" was unwritable as a rule. Four fields: has_created_date, has_changed_date, has_owner, has_changed_by. Scanned as sql.NullInt64 rather than int, which is load-bearing and is what the first cut got wrong. A row without the columns -- any fixture that does not name them -- scans as NULL, NULL into a plain int fails the whole query, and Entities() swallows a query error by yielding nothing. Every rule then sees zero entities and reports zero violations: a green run that checked nothing. That is the same failure shape the catalog-mode bump guards against, reached through the scan instead. Three rule tests went green with the entity table empty before this was fixed. The linter's test fixtures declare their own copy of the entities schema, so adding a column broke them -- three of them INSERTed positionally. They now name their columns, which is why a fourth column will not break them again. That duplication is still a maintenance risk and is worth collapsing onto the real schema separately. Two tests, and the control is what makes them worth having: wiring all four to starlark.Bool(false) makes the first report both entities instead of one and the second report none, so a field that silently reads False everywhere cannot pass. Verified end to end as well -- a QUAL900-style rule over a real 11.14 project names 6 entities, and after `add attribute if not exists CreatedDate: AutoCreatedDate` on one of them it names 5 and stops naming that one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNDe35kDNsMX5cz4Ahn4rk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while writing the MDL equivalent of an "add CreatedDate/ChangedDate to every entity" script: the query anyone would write returns a wrong answer, not an error.
reports every entity as missing
CreatedDateforever, including ones that already have it. Measured before the change: adding the fields and runningrefresh catalog fullleft the result unchanged, andCATALOG.ATTRIBUTESfor that entity still listed only its four ordinary attributes.Why
The catalog wasn't dropping anything. Mendix stores
Owner,ChangedBy,CreatedDateandChangedDateas booleans on the entity's generalization node (NoGeneralization.HasCreatedDate), not as attributes — so they're absent fromCATALOG.ATTRIBUTESby construction.What makes it a trap is that
DESCRIBE ENTITYrenders them in the attribute list asCreatedDate: AutoCreatedDate. The one view a user checks against says they're attributes; the catalog says they don't exist.Commit 1 — four columns on
CATALOG.ENTITIESSimpler than the join, and correct. Fabricating attribute rows was the alternative and is worse — it invents structure the model doesn't have and would disagree with
AttributeCount.Schema version 11 → 12.
CREATE TABLE IF NOT EXISTSdoes not add a column to a cached catalog, so without the bump every query naming one fails withno such columnon exactly the machines that already have a catalog.Commit 2 — the same four on the Starlark
entityWith the data indexed a rule still couldn't read it:
entitycarriedhas_event_handlersandis_externaland nothing about the audit trail, and neither workaround exists — the members aren't attributes, soattributes_for()never yields them andattribute_countexcludes them. Nowhas_created_date,has_changed_date,has_owner,has_changed_by.One subtlety worth flagging for review: they're scanned as
sql.NullInt64, notint. A row without the columns scans as NULL, NULL into a plainintfails the whole query, andEntities()swallows a query error by yielding nothing — so every rule sees zero entities and reports zero violations. A green run that checked nothing. Three rule tests went green with the entity table empty before I caught it.The linter's fixtures declare their own copy of the entities schema and three INSERTed positionally, so a new column broke them; they now name their columns. That duplication is still a maintenance risk, worth collapsing onto the real schema separately.
Verification
Catalog, on a real 11.14 project:
AuthTokenHasCreatedDate/HasChangedDate0/0, then1/1afteradd attribute if not exists CreatedDate: AutoCreatedDate0WHERE HasCreatedDate = 0DESCRIBE CATALOG.ENTITIESLint, end to end with a QUAL900-style rule over the same project: names 6 entities, then 5 after adding the audit trail to one — and stops naming that one.
Controls. Stubbing the catalog values to literal zeros fails
TestEntitiesCarryTheAuditMemberswith the pre-fix symptom. Wiring the Starlark fields tostarlark.Bool(false)makes one test report both entities instead of one and the other report none — so a field that silently reads False everywhere cannot pass.make lint-goandmake testpass.🤖 Generated with Claude Code
https://claude.ai/code/session_01BNDe35kDNsMX5cz4Ahn4rk