feat(sql-insight): creator + generate_qualified_name for SqlInsightFilter and SqlInsightBusinessQuestion - #1030
Open
shubhambatlan wants to merge 3 commits into
Open
Conversation
…usiness question
SqlInsightJoin has both; the other two SqlInsight types had neither, so the only
way to write a filter or a business question was to hand-build the entity and
hand-compute its qualifiedName. That is how a caller ends up duplicating the
miner's row instead of converging on it: the identity has to be byte-identical
and there was no shared implementation to reach for.
Formulas, matching the SQL-Intelligence miner and the UI's authoring path:
filter <columnQN>/filter/md5(operator)
question <datasetQN>/question/md5(questionText)
Both are single-argument md5 with no ordering, so neither has the composite-key
ordering subtlety the join formula carries.
Verified: the rendered methods compile in place; creator() round-trips both
entities with the right attributes and relationship edges; identical inputs
produce identical qualifiedNames; missing column / operator / question text are
rejected; and the qualifiedNames are byte-identical to those written by an
existing REST client whose filter and business-question writes were confirmed
against a live tenant.
Notes on the shape, following the join precedent:
- Filters hang off the COLUMN, so the dataset qualifiedName is derived from the
column's parent rather than accepted as a separate input that could disagree.
- Both the dataset ATTRIBUTE and the column/dataset RELATIONSHIP are written. The
asset page's Usage & Intelligence tab finds filters by the attribute, while the
relationship is what renders the row on the asset; one without the other is
half-visible.
- query_count and unique_users are set to 0: a human-declared insight has no
observed usage and must never claim any.
…erated output The generated assets are committed, so the templates alone would not have shipped anything; this adds the rendered methods and the tests that hold them honest. Tests are written against the identity CONTRACT rather than the implementation: - qualifiedNames are pinned as LITERALS, never recomputed from the formula in the test. The contract is byte-identity with the SQL-Intelligence miner and the UI's authoring path, so an expectation derived from the code under test asserts nothing and would follow a wrong formula without complaint. - an RFC 1321 vector is asserted separately, so "md5 is broken" fails on its own rather than taking every golden down with it and hiding the cause. - convergence AND discrimination: the same insight yields one qualifiedName, a different operator or a reworded question yields a different one. A formula that ignored its inputs would pass a convergence-only test. - the segment literal and the 32-hex lowercase digest are asserted directly, because those are exactly how this identity was got wrong before (a business question once shipped as `/businessQuestion/` plus a truncated sha256). - both anchorings, because a row carrying only one is half-visible: the Usage & Intelligence tab finds these by the dataset attribute, the relationship is what renders the row on the asset. - query_count/unique_users are 0, so a human-declared insight never reports invented popularity. Not covered: updater / trim_to_required, which these types do not implement (nor does SqlInsightJoin). 23 tests pass. The rest of tests/unit/model is unaffected; connection_test's async connector-type failure predates this branch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… output The repo formats with ruff (pre-commit: ruff, ruff --select I --fix, ruff-format), not black/isort. ruff-format joins the filter's default-name expression onto one line; applying that to the generated file alone would have been undone by the next regeneration, so the template carries the same shape. ruff check passes on all five files; 23 tests still pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
✨ Description
SqlInsightJoinshipscreator()andgenerate_qualified_name(). The other twoSqlInsighttypes had neither, so the only way to write a filter or a businessquestion was to hand-build the entity and hand-compute its qualifiedName.
That matters more than the missing convenience. A SQL insight's qualifiedName ends in
an md5 of its own content, and the SQL-Intelligence miner derives it the same way — so
a caller who gets the identity wrong by one byte creates a duplicate of the mined
row rather than converging on it.
SqlInsightJoin.generate_qualified_nameexists sothe SDK, the miner and every other caller share one implementation; this extends that
to the remaining two types.
Formulas, matching the miner's
sql_intelligenceDAG and the UI's authoring path:Both are single-argument md5 with no ordering, so neither carries the composite-key
ordering subtlety the join formula does.
Jira link: n/a — raised from field work on the talk-to-data write-back path
🧩 Type of change
Purely additive: two new methods per type. No existing signature or behaviour changes.
✅ How has this been tested?
23 new unit tests, all passing.
tests/unit/modelis otherwise unaffected —connection_test.py::test_creator_async_rejects_invalid_connector_type_valuefails onmainas well and predates this branch.The tests are written against the identity contract, not the implementation:
test. The contract is byte-identity with a system this SDK does not control, so an
expectation derived from the same code would assert nothing and would follow a wrong
formula without complaint.
fails on its own rather than taking every golden down with it.
different operator or a reworded question yields a different one. A formula that
ignored its inputs would pass a convergence-only test.
because that is exactly how this identity has been got wrong before — a business
question once shipped as
/businessQuestion/plus a truncated sha256.page's Usage & Intelligence tab finds these by the dataset attribute, while the
relationship is what renders the row on the asset itself.
query_count/unique_usersare 0 — a human-declared insight has no queryhistory and must never report invented popularity.
Cross-checked independently: the qualifiedNames these creators produce are
byte-identical to those written by an existing REST client whose filter and
business-question writes were confirmed against a live tenant.
✅ Regenerated and confirmed
The generated assets are committed, so this PR includes them. I have now run the real
generator (
create_typedefs_file.py→class_generator.py) against a live internalsandbox in a throwaway worktree and compared the output:
All four method blocks —
generate_qualified_nameandcreator, on both types —are byte-identical to what is committed here.
Two notes for a reviewer:
regeneration touched 541 files), so only the two
sql_insight_*files were comparedand everything else was discarded. A regeneration against the typedefs this release
is cut from would be the authoritative check; this one confirms the templates render
as expected.
formatting step. The committed files carry the pruned form, matching the existing
sql_insight_join.pyconvention exactly.Formatting is
ruffthroughout (ruff check,ruff --select I --fix,ruff-format)—
ruff checkpasses on all five files. Whereruff-formatreshaped a line in thegenerated output, the same shape was applied to the template, so the next regeneration
does not churn the file back.
📋 Checklist
HISTORY.mdis untouched — it looks curated at release time rather than per PR. Happyto add an entry if that is the convention.
Security review
No new dependencies, no network calls, no credential handling, no logging.
hashlib.md5is used solely to reproduce an existing identity string and is marked
# noqa: S324with that rationale, matching the existing
SqlInsightJoinusage.🔭 Related, not fixed here
While cross-checking the formulas I found that
atlan-frontend'ssqlInsightIdentity.tssorts join column pairs by the formattedSOURCE=JOINEDstring, whereas the miner and this SDK both order by the source column alone. The
two diverge on composite keys where one source column is a prefix of another followed
by a digit (
ORDER_ID/ORDER_ID2), because=is0x3Dand digits are0x30-0x39— so a UI-authored composite join can duplicate a mined row. Single-pairjoins are unaffected. Nothing in this PR is affected; flagging it for whoever owns SQL
Intelligence.