Skip to content

Split selected semantic DAGs into explicit precompute and query projections #740

Description

@milindsrivastava1997

Problem

PrecomputePlan is the backend plan for producing and maintaining summary state. Its serialized executable_dags field currently stores an InstalledPostAsapDag containing the complete selected Planner semantic DAG, including read-time nodes.

The per-node BackendExecutableBinding records where each semantic node belongs, so the backend can avoid executing read-time nodes during maintenance. But that means the content of PrecomputePlan does not match its ownership: it can contain nodes that precomputation must not execute.

This makes the physical-plan artifact and its visualization confusing. A reader cannot tell from the plan boundary alone which DAG nodes construct maintained state and which answer a query.

This is also documented as a known ambiguity in the ASAPPlanner integration design:

The current PrecomputePlan.executable_dags can contain a complete semantic DAG, including query-time nodes such as SummaryEstimate. Bindings may prevent those nodes from running during maintenance, but the artifact and its visualization do not express that ownership clearly.

Concrete example

For:

quantile_over_time(0.99, m[1m])

the selected semantic computation can be understood as:

raw input m
  → build KLL summary
  → read 0.99 quantile from that summary
  → query result

Today, the serialized PrecomputePlan.executable_dags[query_id].document can contain all three semantic nodes:

Semantic node Current binding Actual execution phase
raw input / KeepPreAsap MaintenanceInput maintenance
KLL SummaryAgg Materialization { summary_definition } maintenance; stored state
SummaryEstimate { Quantile { q: 0.99 } } Query { query_node: QueryNodeId(0) } query serving

The final SummaryEstimate is correctly represented in QueryPlan as a QueryPlanNode::SummaryEstimate, but it is also present in the complete semantic DAG retained by PrecomputePlan. The binding map explains the phase distinction only after the reader inspects individual nodes.

Relevant contracts:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions