Share parsed files instead of re-parsing every include - #7
Merged
Merged
Conversation
Parsing `include <BOSL2/std.scad>` costs ~55ms and is ~82% of evaluating a small script against it. The library does not change between renders, so an editor re-rendering, and a docs build rendering a thousand examples, pay that same 55ms over and over. Now a file is parsed once and SHARED. getProgramFromFile() returns borrowed nodes plus the shared trees that keep them alive; the cache is keyed by path and holds one entry per file, replacing it when the file's size or mtime moves. Warm, resolving a script that includes BOSL2 drops from 55ms to 0.12ms. Sharing is what forced the other half of this change: ASTNode no longer carries its own Scope*. It could not -- `include` puts the included nodes in the INCLUDER's scope, so the same BOSL2 node is in a different scope in every script that includes it, and whichever evaluation wrote the field last would corrupt the rest. The pointer moves into a ScopeTable that the root Scope owns, addressed by a (treeId, slot) pair stamped on each node at parse time. That keeps a lookup two loads and no hashing, which matters because the VM reads it on its call path. Nodes are numbered in the constructor from a parse-scoped counter, since there is no generic child walker to number a finished tree with. Only per-FILE parses are cached, never a resolved program: a file is spliced in at most once per resolution, so caching std.scad already-resolved would double its statements in a script that also includes something else depending on it. Re-running the resolution is only pointer pushes. buildScopes() keeps its signature -- it creates the table and hands it to the root Scope -- so the ~170 existing call sites are untouched. Reads go through the new scopeOf(scope, node). 648 parser tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Sep 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Parsing
include <BOSL2/std.scad>costs ~55 ms and is ~82% of evaluating a small script against it. The library does not change between renders, so an editor re-rendering — and a docs build rendering a thousand examples — pay that 55 ms over and over.Sharing forced the second half of this change
ASTNodeno longer carries its ownScope*. It could not:includeputs the included nodes in the includer's scope, so the same BOSL2 node is in a different scope in every script that includes it, and whichever evaluation wrote the field last would corrupt the rest.The pointer moves into a
ScopeTablethat the rootScopeowns, addressed by a(treeId, slot)pair stamped on each node at parse time. A lookup stays two loads and no hashing — which matters, because the VM reads it on its call path. Nodes are numbered in the constructor from a parse-scoped counter, since there is no generic child walker to number a finished tree with.Only per-file parses are cached
Never a resolved program. A file is spliced in at most once per resolution (
visited), so cachingstd.scadalready-resolved would double its statements in a script that also includes something else depending on it. Re-running the resolution is only pointer pushes. There's a test for exactly this.Churn kept low
buildScopes()keeps its signature — it creates the table and hands it to the rootScope— so the ~170 existing call sites are untouched. Reads go through a newscopeOf(scope, node).The cache holds one entry per file, replaced when size or mtime moves, so an editor saving repeatedly doesn't accumulate a copy of every version.
Verified
🤖 Generated with Claude Code