Skip to content

Share parsed files instead of re-parsing every include - #7

Merged
revarbat merged 1 commit into
mainfrom
shared-ast-cache
Sep 6, 2026
Merged

revarbat merged 1 commit into
mainfrom
shared-ast-cache

Conversation

@revarbat

@revarbat revarbat commented Sep 6, 2026

Copy link
Copy Markdown
Member

Parsing include <BOSL2/std.scad> costs ~55 ms and is ~82% of evaluating a small script against it. The library does not change between renders, so an editor re-rendering — and a docs build rendering a thousand examples — pay that 55 ms over and over.

before after (warm)
Resolve a script including BOSL2 55.4 ms 0.12 ms

Sharing forced the second half of this change

ASTNode no longer carries its own Scope*. It could not: include puts the included nodes in the includer's scope, so the same BOSL2 node is in a different scope in every script that includes it, and whichever evaluation wrote the field last would corrupt the rest.

The pointer moves into a ScopeTable that the root Scope owns, addressed by a (treeId, slot) pair stamped on each node at parse time. A lookup stays two loads and no hashing — which matters, because the VM reads it on its call path. Nodes are numbered in the constructor from a parse-scoped counter, since there is no generic child walker to number a finished tree with.

Only per-file parses are cached

Never a resolved program. A file is spliced in at most once per resolution (visited), so caching std.scad already-resolved would double its statements in a script that also includes something else depending on it. Re-running the resolution is only pointer pushes. There's a test for exactly this.

Churn kept low

buildScopes() keeps its signature — it creates the table and hands it to the root Scope — so the ~170 existing call sites are untouched. Reads go through a new scopeOf(scope, node).

The cache holds one entry per file, replaced when size or mtime moves, so an editor saving repeatedly doesn't accumulate a copy of every version.

Verified

  • 648 parser tests pass
  • Downstream: 1153 evaluator tests, 31 binding tests, and BOSL2's 909 regression tests

🤖 Generated with Claude Code

Parsing `include <BOSL2/std.scad>` costs ~55ms and is ~82% of evaluating a
small script against it. The library does not change between renders, so an
editor re-rendering, and a docs build rendering a thousand examples, pay
that same 55ms over and over.

Now a file is parsed once and SHARED. getProgramFromFile() returns borrowed
nodes plus the shared trees that keep them alive; the cache is keyed by path
and holds one entry per file, replacing it when the file's size or mtime
moves. Warm, resolving a script that includes BOSL2 drops from 55ms to
0.12ms.

Sharing is what forced the other half of this change: ASTNode no longer
carries its own Scope*. It could not -- `include` puts the included nodes in
the INCLUDER's scope, so the same BOSL2 node is in a different scope in
every script that includes it, and whichever evaluation wrote the field last
would corrupt the rest. The pointer moves into a ScopeTable that the root
Scope owns, addressed by a (treeId, slot) pair stamped on each node at parse
time. That keeps a lookup two loads and no hashing, which matters because
the VM reads it on its call path. Nodes are numbered in the constructor from
a parse-scoped counter, since there is no generic child walker to number a
finished tree with.

Only per-FILE parses are cached, never a resolved program: a file is spliced
in at most once per resolution, so caching std.scad already-resolved would
double its statements in a script that also includes something else
depending on it. Re-running the resolution is only pointer pushes.

buildScopes() keeps its signature -- it creates the table and hands it to
the root Scope -- so the ~170 existing call sites are untouched. Reads go
through the new scopeOf(scope, node).

648 parser tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@revarbat
revarbat merged commit e4a75fc into main Sep 6, 2026
3 checks passed
@revarbat
revarbat deleted the shared-ast-cache branch September 6, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant