RFC: formal DBML spec, layer 1 grammar and conformance harness

[MFornander]

RFC, not seeking merge yet. This draft exists to discuss the approach and to make the diff easy to read. Inline comments on the grammars are very welcome.

Summary

DBML has no formal specification. packages/dbml-parse is the de facto definition, CONTRIBUTING.md still points at PEG.js, and the only PEG grammar in the repo (packages/dbml-core/src/parse/deprecated/dbml/parser.pegjs) carries a header saying it is deprecated and should not be maintained.

This PR proposes a layered spec that mirrors the parser's own layers, and lands the first layer in two notations so the notation choice can be judged on evidence:

• spec/README.md: the layered design (generic syntax, per-element semantics, interpreted model), the authority rule, a one-page summary of the generic syntax, and a side-by-side comparison of the two notations.
• spec/dbml-syntax.peggy: an executable PEG grammar for Layer 1. It reproduces the SyntaxTokenKind and SyntaxNodeKind sets exactly, including the three trivia-sensitive rules (index vs new argument, call vs new line, argument spacing), the Pratt binding powers, comma expressions with empty slots, attribute lists, use declarations, and the function-application to nested-element reinterpretation. Element keywords are plain identifiers at this layer.
• spec/antlr/DbmlLexer.g4 and spec/antlr/DbmlParser.g4: the same specification as a two-stage ANTLR 4 grammar, generated with antlr-ng (a Node port of the ANTLR tool, no Java) for the antlr4 JavaScript runtime that @dbml/core already uses for its SQL importers.
• packages/dbml-parse/__tests__/conformance/: the authority mechanism. syntax.test.ts runs the reference lexer+parser and each spec parser over every __tests__/snapshots/**/input/*.dbml and asserts agreement on accept/reject and on the normalised node-kind tree. properties.test.ts does the same over the existing fast-check arbitraries. Both grammars are compiled at test time; no generated code is committed.
• spec/DISAGREEMENTS.md: every case where spec and parser disagree, each with a verdict. Every entry is pinned by a test for each notation, so a fix on any side fails a test and forces the document to change.

Nothing under packages/dbml-parse/src changes. The spec restates behaviour; it does not alter it.

Conformance results on this branch

 syntax.test.ts      286 passed   (143 per notation)
 properties.test.ts   38 passed   (19 per notation)

Check	peggy	ANTLR
Snapshot inputs, all 7 snapshot directories (122 files)	121 agree on verdict and full tree, 1 recorded disagreement	identical
Property tests: 500 whole-schema runs plus 17 element and malformed generators	no disagreements	identical
Existing @dbml/parse suite	unchanged, 80 files / 2013 tests; conformance tests are excluded from the default run

The five recorded disagreements are all lexer edge cases (Note: 12. at end of file is rejected; 1a and 1.a at end of file lex as numbers; \u accepts non-hex alphanumerics; letters outside the Basic Multilingual Plane are rejected). Both notations reproduce them identically. Details and verdicts are in spec/DISAGREEMENTS.md.

Open questions

1. Notation. Both renderings pass the same corpus, so the choice is about readability and maintenance. The trade-off as measured on this branch (details in spec/README.md):
   • peggy: one 552-line file; scannerless, so trivia is threaded through the rules by hand; about 116 lines of JavaScript in the initializer plus one-line tree-building actions on most rules; compiled in memory with a single dev dependency.
   • ANTLR: a 160-line lexer with no code and a 336-line parser with a 71-line members block and 17 predicates; trivia on the hidden channel, which matches the reference lexer's model closely; tree shaping lives in a 170-line converter in the test harness; needs antlr-ng and antlr4 as dev dependencies and a generation step. It is the toolchain dbml-core already uses.
     Which one should the spec standardise on? Keeping both long-term doubles the maintenance cost and is not proposed.
2. Location. Spec files currently live in a top-level spec/ directory with tests in packages/dbml-parse. Should they instead live inside packages/dbml-parse, or become a new @dbml/spec package that downstream tools can depend on?

Planned follow-ups (separate PRs)

• spec/output/database.schema.json: JSON Schema of the interpreted Database model, validated against snapshots/interpreter/output/*.out.json.
• spec/elements/<element>.schema.json: one per element, starting with Table, Ref, Enum, then Dep, each with a conformance test against the corresponding validate.ts.
• Flip the conformance job to blocking once DISAGREEMENTS.md is empty, and generate dbml-homepage/docs/syntax/formal-grammar.md from spec/.

Issue

None yet. Happy to open one if you would rather track the discussion there.

Lasting Changes (Technical)

• Added peggy, antlr-ng and antlr4 as devDependencies of @dbml/parse only. peggy 5 requires Node 20 or newer; both package.json files declare >=18. CI runs Node 22, and consumers are unaffected since these are dev-only, but flagging it.
• Added the test:conformance script and vitest.conformance.config.ts in @dbml/parse, with a global setup that compiles spec/antlr/*.g4 into __tests__/conformance/generated/ (git-ignored). The default vitest run now excludes __tests__/conformance/**.
• Added a non-blocking spec-conformance job to .github/workflows/test.yml (continue-on-error: true).
• Rewrote the stale PEG.js paragraph in CONTRIBUTING.md to point at packages/dbml-parse, spec/, and the ANTLR importers. Not touched here, but noticed: the label table in CONTRIBUTING.md is missing PR: Internal and PR: Dependencies Update, and lists the Refactor emoji as :poop: where the repo label uses :broom:.

Labels

I do not have triage rights on this repo. Suggested labels: PR: New Feature :rocket: (or PR: Internal :house_with_garden: if you consider the spec internal tooling) and pkg: parse.

Checklist

• Documentation (if necessary)
• Updated dbml-homepage/static/llms.txt (not applicable: no user-facing syntax or docs changed)
• Lint Checks Passed
• Unit Tests Passed
• Coverage Tests Passed (runs in CI; conformance tests are excluded from coverage by design)
• Integration Tests Passed (runs in CI; no change to the packages it exercises)
• Code Review

🤖 Generated with Claude Code