Skip to content

docs: reorganize newcomer documentation - #439

Merged
zzylol merged 10 commits into
mainfrom
docs/newcomer-documentation
Sep 17, 2026
Merged

zzylol merged 10 commits into
mainfrom
docs/newcomer-documentation

Conversation

@milindsrivastava1997

@milindsrivastava1997 milindsrivastava1997 commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

Why

Readers need clear documentation locations and descriptions that match the implemented Planner. The documentation should retain its original explanations, chapter order, diagrams and examples while correcting stale information.

What changed

  • Retain the PR's design/developer/user directory organization and reading indexes.
  • Preserve the original mapping design, code architecture, contracts, extension-guide chapters, physical-operator catalog and CLI guide structure. Update their information in place rather than splitting or replacing these documents.
  • Keep DDSketch ratio certification under ASAP-aware mapping proposals. Return the accuracy implementation guide to docs/develop_docs/end-to-end-accuracy-guarantees.md, with updated cross-links.
  • Update replacement/provenance contracts, registered strategies, accuracy rejection, optional coordinated selection and materialization, and sketch-library dependency descriptions to match the implementation.
  • Keep the Post-ASAP concept document in its original list-based format, with all 11 current SummaryExpr variants and an explanation of why SummaryCreate/SummaryInsert are not separate current variants.
  • Preserve the library API guide from docs: explain Planner workflows and public library controls #440 and the all-candidate CLI behavior from feat(devtools): show all post-ASAP candidates #435. Keep corpus verification commands in the existing CLI guide and repair moved links.
  • Clarify implemented scope versus remaining work in proposal status notes.

The subsequent document-splitting approach has been withdrawn. The final changes retain existing document structure and update facts, examples and links where needed. Rebased onto main at 04e5a4e. Rust changes only update documentation/schema paths; planner behavior is unchanged.

Before this PR

Documentation locations mixed audiences, and several references described older node/strategy contracts or overstated implementation limitations. The original chapter structures and examples still contained useful context.

After this PR

Readers can follow audience-specific navigation while retaining the original chapters, diagrams and examples. The existing sections explain current candidate forms, accuracy certification and selection boundaries; the Post-ASAP node list is complete and agrees with the code.

Validation

  • All original headings and their order preserved in the seven restored core documents.
  • Post-ASAP conceptual catalog matches all 11 SummaryExpr variants.
  • Local link/anchor check: 77 Markdown files, all 204 local links resolve.
  • git diff --check passed.
  • Earlier validation on this PR passed cargo fmt, the devtools binary build, SQL/PromQL file/stdin inspection examples, DAG-export JSON checks and the relocated-schema unit test.
  • After the latest rebase, cargo test -p asap-devtools --bin show_post_asap_ir passed.
  • The structure-restoration commit changes documentation only; runtime tests were not repeated. Full workspace tests/clippy were not run, and pre-commit is not installed in this environment.

@zzylol
zzylol force-pushed the docs/newcomer-documentation branch from 052f8c6 to 0db8705 Compare September 17, 2026 18:42
@zzylol
zzylol force-pushed the docs/newcomer-documentation branch from 0db8705 to ca5f105 Compare September 17, 2026 18:44
@zzylol

zzylol commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

@milindsrivastava1997 Clarifying the Post-ASAP node lists, and correcting an omission in this documentation reorganization:

The old docs/design_docs/post-asap-ir.md on main is an earlier design sketch. This PR replaces it with docs/design_docs/concepts/post-asap-ir.md and a developer reference aligned with the existing SummaryExpr implementation. This PR does not change the IR implementation.

My initial rewrite compressed merge/join/subtract/delete into a generic sentence. Although the developer reference still listed them, that made the conceptual page incomplete as a node catalog. Fixed in 7f061e2: the conceptual page now explicitly lists all 11 current variants and their semantics, including SummaryEstimate, SummaryMerge, SummarySubtract, SummaryDelete, and SummaryJoin. Both documentation catalogs were checked against the enum; nothing from the current node vocabulary is omitted.

Why no separate SummaryCreate / SummaryInsert nodes?

Those names occur in the earlier design sketch, but are not variants of the current SummaryExpr enum. SummaryAgg describes the state-producing computation: child input, selected family/parameters, update input (SummaryUpdate), reduction, and grouping layout. It does not split state initialization and each incoming-record update into separate logical DAG nodes.

The summary-maintenance lifecycle layer separately models when that state is created, retained, shared, updated, and retired. It considers ephemeral, prepared, shared, and continuously maintained lifecycles; direct build versus incremental maintenance is another deployment choice, subject to capabilities and evidence. Downstream physical binding/runtime implements the actual build and update operations.

Creation and insertion/update behavior therefore still exists through the summary aggregate and its maintenance/physical realization. This is not a one-to-one rename of SummaryCreate/SummaryInsert to SummaryAgg, nor a claim that every family supports incremental updates. The new conceptual page now explains this distinction explicitly, while retaining a complete list of the other current nodes. The developer reference adds construction, validation, and export obligations.

Update after the documentation-structure review (5e81b67): The final version preserves the original documents and chapter order instead of splitting them. The Post-ASAP document now uses the original list-based structure, contains all 11 current variants, and retains the create/insert explanation above. The separately added developer node-reference file has been withdrawn; the earlier links in this comment describe the intermediate revision, not the final organization.

@zzylol

zzylol commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

@milindsrivastava1997 I updated some status after comparing the doc with the code, and kept your reorganization structure. So I will merge it.

@zzylol
zzylol merged commit ba1c443 into main Sep 17, 2026
4 checks passed
@zzylol
zzylol deleted the docs/newcomer-documentation branch September 17, 2026 23:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants