Skip to content

ClickHouse virtual catalog is not stripped from CREATE VIEW source queries #5938

Description

@mday-io

Describe the bug

In a multi-gateway project using ClickHouse's virtual-catalog support from
#5826, SQLMesh strips the synthetic catalog from the target of a virtual-layer
CREATE VIEW, but leaves it on the physical view referenced by the SELECT.

ClickHouse receives an invalid three-part source name:

CREATE OR REPLACE VIEW "my_db"."connection_test__dev" ON CLUSTER "my_cluster"
AS SELECT *
FROM "__clickhouse_gw__"."my_db"."my_db__connection_test__1234567890"

and rejects it because ClickHouse supports only database.table names.

To reproduce

  1. Configure a project with a catalog-aware default gateway and a ClickHouse
    secondary gateway.
  2. Enable gateway_managed_virtual_layer.
  3. Define a ClickHouse VIEW model using gateway clickhouse_gw.
  4. Run sqlmesh plan dev --auto-apply.

The physical view is created successfully. Virtual-layer promotion then fails
with ClickHouse error code 62 when SQLMesh executes the three-part source
reference shown above.

Expected behavior

SQLMesh should remove the configured synthetic virtual catalog from every
ClickHouse table reference before sending view DDL to ClickHouse. The generated
query should be equivalent to:

CREATE OR REPLACE VIEW "my_db"."connection_test__dev" ON CLUSTER "my_cluster"
AS SELECT *
FROM "my_db"."my_db__connection_test__1234567890"

Likely cause

The @set_catalog() handling on the inherited create_view() method normalizes
the view_name argument, but not tables nested inside the query_or_df
expression. The snapshot evaluator supplies the physical source using its
three-level virtual FQN, so the synthetic catalog reaches ClickHouse.

A local adapter override that copied the query expression and removed only the
adapter's configured virtual catalog from nested exp.Table nodes allowed the
same plan to promote successfully. An unexpected catalog was still rejected.

Impact

Gateway-managed virtual-layer promotion cannot complete for ClickHouse models
in affected multi-gateway projects. Physical objects may be created before the
promotion failure, leaving a partially applied plan to clean up.

Environment

  • SQLMesh: 0.236.0
  • ClickHouse: 25.8
  • ClickHouse adapter in cluster mode

Related work

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugSomething isn't workingEngine: ClickhouseIssues related to Clickhouse

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions