Skip to content

feat(caching): support incremental builds - #1010

Open
avivkeller wants to merge 3 commits into
mainfrom
feat/incremental-cache
Open

feat(caching): support incremental builds#1010
avivkeller wants to merge 3 commits into
mainfrom
feat/incremental-cache

Conversation

@avivkeller

@avivkeller avivkeller commented Aug 7, 2026

Copy link
Copy Markdown
Member

No description provided.

@avivkeller
avivkeller requested a review from a team as a code owner August 7, 2026 15:44
@cursor

cursor Bot commented Aug 7, 2026

Copy link
Copy Markdown

You have used all Bugbot PR reviews included in your team's free trial on GitHub.

To continue using Bugbot reviews, enable Bugbot for your team in the Cursor dashboard.

@vercel

vercel Bot commented Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
api-docs-tooling Ready Ready Preview Aug 8, 2026 12:32am

Request Review

@cursor

cursor Bot commented Aug 7, 2026

Copy link
Copy Markdown

You have used all Bugbot PR reviews included in your team's free trial on GitHub.

To continue using Bugbot reviews, enable Bugbot for your team in the Cursor dashboard.

@avivkeller
avivkeller force-pushed the feat/incremental-cache branch from a629257 to 6e4d3d6 Compare August 7, 2026 16:04
@cursor

cursor Bot commented Aug 7, 2026

Copy link
Copy Markdown

You have used all Bugbot PR reviews included in your team's free trial on GitHub.

To continue using Bugbot reviews, enable Bugbot for your team in the Cursor dashboard.

@codecov

codecov Bot commented Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 67.86607% with 643 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.65%. Comparing base (740b613) to head (103bd73).

Files with missing lines Patch % Lines
packages/core/bin/commands/watch.mjs 0.00% 149 Missing ⚠️
packages/core/src/cache/index.mjs 70.34% 102 Missing ⚠️
packages/legacy/src/legacy-html/generate.mjs 20.83% 95 Missing ⚠️
packages/core/bin/utils.mjs 0.00% 81 Missing ⚠️
packages/react/src/jsx-ast/generate.mjs 51.07% 68 Missing ⚠️
packages/react/src/html/utils/processing.mjs 67.68% 53 Missing ⚠️
packages/core/src/utils/file.mjs 50.00% 24 Missing ⚠️
packages/core/src/cache/store.mjs 92.68% 12 Missing ⚠️
packages/core/src/utils/unist.mjs 53.84% 12 Missing ⚠️
packages/core/src/generators/metadata/generate.mjs 0.00% 7 Missing ⚠️
... and 12 more
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1010      +/-   ##
==========================================
- Coverage   89.44%   87.65%   -1.80%     
==========================================
  Files         203      214      +11     
  Lines       18750    20517    +1767     
  Branches     1749     1876     +127     
==========================================
+ Hits        16771    17984    +1213     
- Misses       1972     2524     +552     
- Partials        7        9       +2

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

api-links Generator

Performance estimate (single CI run)

  • Generation time: 6.3% slower (1.43 s → 1.52 s)
  • Peak memory: 3.1% lower (357.06 MB → 345.95 MB)

legacy-html Generator

Performance estimate (single CI run)

  • Generation time: 1.3% slower (19.78 s → 20.03 s)
  • Peak memory: 3.8% higher (2.05 GB → 2.13 GB)

legacy-json Generator

Performance estimate (single CI run)

  • Generation time: 19.0% faster (8.83 s → 7.15 s)
  • Peak memory: 10.7% higher (1.56 GB → 1.73 GB)

llms-txt Generator

Performance estimate (single CI run)

  • Generation time: 8.0% faster (8.67 s → 7.98 s)
  • Peak memory: 19.1% higher (1.43 GB → 1.70 GB)

orama-db Generator

Output size: 1 file changed · net +13.97 KB

File size details
File Main PR Change
orama-db.json 9.07 MB 9.08 MB +13.97 KB (+0.2%)

Performance estimate (single CI run)

  • Generation time: 5.4% slower (8.17 s → 8.61 s)
  • Peak memory: 6.7% higher (1.64 GB → 1.75 GB)

web Generator

Output size: 200 files changed · net -31.51 MB

File size details
File Main PR Change
all.html 31.51 MB — -31.51 MB (-100.0%)
assets/SearchBox-HH0giLJU.js 84.66 KB — -84.66 KB (-100.0%)
assets/SearchBox-nhKtQ-aJ.js — 84.66 KB +84.66 KB
assets/dist-Dwppe9BV.js 30.40 KB — -30.40 KB (-100.0%)
assets/dist-BqXBNLhZ.js — 30.40 KB +30.40 KB
assets/SideBar-BLca9X0j.js 25.25 KB — -25.25 KB (-100.0%)
assets/SideBar-DfcHyd63.js — 25.25 KB +25.25 KB
assets/runtime-BQmIUe4H.js 19.67 KB — -19.67 KB (-100.0%)
assets/runtime-BTw5MQsN.js — 19.67 KB +19.67 KB
assets/Combination-DMDCwc-k.js 16.10 KB — -16.10 KB (-100.0%)
assets/Combination-44yqdQLj.js — 16.10 KB +16.10 KB
assets/compat.module-ClijkQGx.js 13.98 KB — -13.98 KB (-100.0%)
assets/compat.module-RlhC4PCC.js — 13.98 KB +13.98 KB
assets/ThemeToggle-nAxhyweS.js 13.56 KB — -13.56 KB (-100.0%)
assets/ThemeToggle-vS1eaEw7.js — 13.56 KB +13.56 KB
assets/dist-BJf1BX_X.js 10.17 KB — -10.17 KB (-100.0%)
assets/dist-DDBUuHyc.js — 10.17 KB +10.17 KB
assets/Layout-BxOeyXZ4.js 7.97 KB — -7.97 KB (-100.0%)
assets/Layout-Uz1QJ76i.js — 7.97 KB +7.97 KB
assets/Tooltip-BJFyCEzU.js 7.86 KB — -7.86 KB (-100.0%)
assets/Tooltip-Fn9_4XhZ.js — 7.86 KB +7.86 KB
assets/dist-BdrqGdF9.js 6.95 KB — -6.95 KB (-100.0%)
assets/dist-Bi5BnIef.js — 6.95 KB +6.95 KB
assets/withIsland-DvwC_S4u.js 4.15 KB — -4.15 KB (-100.0%)
assets/withIsland-D85Ih1E9.js — 4.15 KB +4.15 KB
assets/dist-BLtz5aSc.js 3.91 KB — -3.91 KB (-100.0%)
assets/dist-BZeH5DJ7.js — 3.91 KB +3.91 KB
assets/CodeTabs-BATDFBz3.js 3.80 KB — -3.80 KB (-100.0%)
assets/CodeTabs-CGRk7JGY.js — 3.80 KB +3.80 KB
assets/CodeBox-Cm-pxJji.js 3.33 KB — -3.33 KB (-100.0%)
assets/CodeBox-DodbSgoO.js — 3.33 KB +3.33 KB
assets/404-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/addons-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/all-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/assert-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/async_context-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/async_hooks-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/buffer-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/child_process-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/cli-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/cluster-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/console-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/crypto-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/debugger-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/deprecations-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/dgram-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/diagnostics_channel-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/dns-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/documentation-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/domain-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/dtls-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/embedding-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/environment_variables-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/errors-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/esm-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/events-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/ffi-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/fs-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/globals-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/http-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/http2-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/https-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/index-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/inspector-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/intl-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/module-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/modules-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/n-api-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/net-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/os-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/packages-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/path-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/perf_hooks-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/permissions-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/process-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/punycode-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/querystring-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/quic-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/readline-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/repl-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/report-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/single-executable-applications-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/sqlite-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/stream-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/stream_iter-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/string_decoder-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/synopsis-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/test-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/timers-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/tls-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/tracing-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/tty-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/typescript-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/url-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/util-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/v8-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/vfs-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/vm-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/wasi-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/webcrypto-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/webstreams-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/worker_threads-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/zlib-ByRYh0OC.js 2.27 KB — -2.27 KB (-100.0%)
assets/404-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/addons-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/assert-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/async_context-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/async_hooks-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/buffer-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/child_process-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/cli-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/cluster-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/console-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/crypto-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/debugger-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/deprecations-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/dgram-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/diagnostics_channel-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/dns-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/documentation-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/domain-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/dtls-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/embedding-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/environment_variables-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/errors-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/esm-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/events-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/ffi-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/fs-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/globals-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/http-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/http2-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/https-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/index-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/inspector-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/intl-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/module-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/modules-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/n-api-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/net-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/os-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/packages-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/path-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/perf_hooks-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/permissions-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/process-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/punycode-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/querystring-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/quic-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/readline-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/repl-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/report-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/single-executable-applications-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/sqlite-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/stream-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/stream_iter-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/string_decoder-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/synopsis-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/test-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/timers-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/tls-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/tracing-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/tty-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/typescript-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/url-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/util-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/v8-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/vfs-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/vm-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/wasi-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/webcrypto-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/webstreams-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/worker_threads-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/zlib-Cd-9AAwS.js — 2.27 KB +2.27 KB
assets/FunctionSignature-Df4fosQI.js 2.19 KB — -2.19 KB (-100.0%)
assets/FunctionSignature-DFpWz3sl.js — 2.19 KB +2.19 KB
assets/Banner-CYPXYjoI.js 2.08 KB — -2.08 KB (-100.0%)
assets/Banner-Cjs9P7Z8.js — 2.08 KB +2.08 KB
assets/ChangeHistory--MacoUzE.js 1.71 KB — -1.71 KB (-100.0%)
assets/ChangeHistory-CwvHaAVY.js — 1.71 KB +1.71 KB
assets/DataTag-C3q9klfV.js 841.00 B — -841.00 B (-100.0%)
assets/DataTag-BI6Lntl1.js — 841.00 B +841.00 B
assets/classnames-DGcsN1jh.js 752.00 B — -752.00 B (-100.0%)
assets/classnames-BekBWDid.js — 752.00 B +752.00 B
assets/Badge-Po8dyGcw.js 604.00 B — -604.00 B (-100.0%)
assets/Badge-BZrdn3wo.js — 604.00 B +604.00 B
assets/AlertBox-D8T5g2fF.js 588.00 B — -588.00 B (-100.0%)
assets/AlertBox-C3TWKAJp.js — 588.00 B +588.00 B
assets/ArrowUpRightIcon-BXGdvwFe.js 587.00 B — -587.00 B (-100.0%)
assets/ArrowUpRightIcon-CW6v0M5v.js — 587.00 B +587.00 B
assets/CodeBracketIcon-DJ0YtqgX.js 520.00 B — -520.00 B (-100.0%)
assets/CodeBracketIcon-D3IBvdvq.js — 520.00 B +520.00 B
assets/dist-KNcjaqkd.js 484.00 B — -484.00 B (-100.0%)
assets/dist-mlUTRQSP.js — 484.00 B +484.00 B
assets/ChevronDownIcon-B0JRzmWj.js 476.00 B — -476.00 B (-100.0%)
assets/ChevronDownIcon-CK2kYDw9.js — 476.00 B +476.00 B
assets/jsx-runtime-CzI9thnQ.js 460.00 B — -460.00 B (-100.0%)
assets/jsx-runtime-Bs2mLgb1.js — 460.00 B +460.00 B
assets/Blockquote-BYdI9zAB.js 168.00 B — -168.00 B (-100.0%)
assets/Blockquote-Dfdw8Dtg.js — 168.00 B +168.00 B

Performance estimate (single CI run)

  • Generation time: 55.2% faster (86.02 s → 38.55 s)
  • Peak memory: 29.9% lower (4.71 GB → 3.30 GB)

ovflowd
ovflowd reviewed Aug 7, 2026
View reviewed changes
Comment thread packages/core/bin/commands/generate.mjs Outdated
@cursor

cursor Bot commented Aug 8, 2026

Copy link
Copy Markdown

You have used all Bugbot PR reviews included in your team's free trial on GitHub.

To continue using Bugbot reviews, enable Bugbot for your team in the Cursor dashboard.

@avivkeller
avivkeller requested a review from ovflowd August 9, 2026 11:55
ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/core/bin/commands/watch.mjs
* @param {() => Promise<void>} task - The work to run
* @returns {{schedule: () => void, cancel: () => void}}
*/
const createScheduler = task => {

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo: this should live on a dedicated scheduler utility file

ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/core/bin/utils.mjs
* @param {string} name - The command name
* @returns {import('commander').Command} The command, for chaining
*/
export const createCommonCommand = name =>

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: probably live on a deidcated command uitils file? avoid bloating utils.mjs with all sort of unrelated different kind of utils?

ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/core/src/cache/hash.mjs
@@ -0,0 +1,54 @@
'use strict';

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we're reinventing the wheel here, can't we use built-in tools to Node?

@avivkeller avivkeller Aug 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a wrapper around import { hash } from 'node:crypto';, so we are using built in tools.

ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/core/src/cache/index.mjs
export const resolveCacheDir = explicit =>
resolve(
explicit ||
(existsSync('node_modules')

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: will this always correctly resolve to the right node_modules folder? From which context is existsSync called from?

@avivkeller avivkeller Aug 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

existsSync('node_modules') will find the node_modules folder of the caller. I can replace this with a dependency, but not sure if it's worth the dependency install for ~10 lines of code

ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/core/src/cache/index.mjs
try {
const path = join(outputDir, rel);

if ((await stat(path)).size !== record.size) {

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: separate await from statement to declaration

ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/core/src/cache/index.mjs
const store = createStore(dir);
const manifest = await loadManifest(dir);

await resolveTypeMap(configuration);

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: why we need specific handling just for the type map?

@avivkeller avivkeller Aug 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type map can either be an object or a file path, so we need to resolve it

ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/core/src/cache/index.mjs

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: I really enjoy the code in this file, but just wondering if it makes sense for us to create and host a whole cache system.

@avivkeller avivkeller Aug 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know of a dependency that does this for us?

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No idea either, just thinking out loud.

ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/core/src/cache/manifest.mjs

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: as amount of files increase, how much of time and cpu are we wasting by reading all these files, computing salts/hashes and comparing them against current ones in the pipeline? Are the savings of incremental builds that great?

@avivkeller avivkeller Aug 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the savings of incremental builds that great?

Incremental builds allow us to host a dev server, and allow re-builds to take ~10s rather than ~40s+. If we have no intention of this functionality, then it's not really needed

ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/core/src/cache/salt.mjs
* existing entry; releases that don't change output leave caches valid. When
* hacking on generator code locally, build with `--force`.
*/
export const CACHE_SCHEMA = 1;

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should this be somewhere else?

ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/core/src/cache/store.mjs
withFileTypes: true,
}).catch(() => []);

await Promise.all(

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little bit wacky, wondering if there's a cleaner way of doing this? Also I wonder if cache cleanup should be the job of doc-kit itself... What other tools such as turborepo/next.js do? Do they clean their own cache? And how? Can you compare impls?

ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/core/src/cache/types.d.ts

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really can't imagine that an existing cache system for the exact sort of thing we're doing doesn't exist. So many frameworks/tools out there need to do the exact same thing. Could you please do me a favor, maybe use Claude, and see what's out there? I really would appreciate not needing to maintain all of this extra code.

ovflowd
ovflowd reviewed Aug 9, 2026
View reviewed changes
Comment thread packages/react/src/sitemap/generate.mjs
const urlset = apiPages
.map(page =>
entryTemplate
(page.lastmod

@ovflowd ovflowd Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: is this relevant? it feels like it could be done on its own? (separate PR)

ovflowd
ovflowd requested changes Aug 9, 2026
View reviewed changes

@ovflowd ovflowd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is doing so much (watch commands, changes on generators that seem unrelated + creation of a whole caching system) although I like the idea of incremental build supports, how many seconds of improvements are we talking about (assume only fs.md and http.md) got touched?

I want to understand what benefits this is bringing, and if we can decouple the PR a tiny bit. It's really hard to appropriately review such a big PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ovflowd ovflowd ovflowd requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@avivkeller @ovflowd