Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# tsv changelog

Covers the npm packages published from this repo — `@fuzdev/tsv_format_wasm`,
`@fuzdev/tsv_parse_wasm`, and `@fuzdev/tsv_wasm`, plus the native N-API set
Covers the npm packages published from this repo — `@fuzdev/tsv-format-wasm`,
`@fuzdev/tsv-parse-wasm`, and `@fuzdev/tsv-wasm`, plus the native N-API set
(`@fuzdev/tsv` and its `@fuzdev/tsv-<triple>` platform packages). All move
together at the `Cargo.toml [workspace.package]` version.
Each `## Unreleased` section must be non-empty and carry a
Expand All @@ -11,8 +11,15 @@ Each `## Unreleased` section must be non-empty and carry a
`## Unreleased` (reset to `bump: patch`) for the next cycle.

## Unreleased
<!-- bump: patch -->
<!-- bump: minor -->

- **breaking** chore: the three WASM packages are renamed to kebab-case — `@fuzdev/tsv_wasm` →
`@fuzdev/tsv-wasm`, `@fuzdev/tsv_format_wasm` → `@fuzdev/tsv-format-wasm`,
`@fuzdev/tsv_parse_wasm` → `@fuzdev/tsv-parse-wasm` — matching the `@fuzdev/tsv-<triple>` platform
packages, the `tsv-<triple>` release assets and the `tsv-format` VS Code extension, so tsv's whole
npm surface spells one way; the snake_case names stop at 0.3.x and are deprecated on npm with a
pointer to their replacement. Exports, options and the `tsv` bin are unchanged; the Rust crates
keep their `tsv_*` names
- fix: restore the CSS parse speed lost in 0.3.0 — a declaration value is ruled out for operator
splitting before its run is tokenized ([#1037](https://github.com/fuzdev/tsv/pull/1037))

Expand Down
16 changes: 9 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ Three binding crates for different use cases:
- `tsv_wasm` (wasm-bindgen) — browser, Deno, Node; output: `.wasm` module (format / parse / all variants via cargo features)
- `tsv_napi` (napi-rs) — Node.js / Bun native addon (`libtsv_napi.*`, loaded via `process.dlopen`). Builds with the `napi` profile (`release` + `panic = "unwind"` → `target/napi/`; every export is `catch_unwind`, so a panic throws a JS error instead of aborting the host). The npm surface — the bare `@fuzdev/tsv` loader over per-platform `@fuzdev/tsv-<triple>` packages, wasm-API-parity by contract; each platform package also ships the `tsv_cli` binary, which the loader's `tsv` bin execs (`npx tsv` = the native CLI) — is staged by `deno task build:napi:packages` (which also builds `tsv_cli --release`) and tested per OS by `test:napi:npm`; the cross-platform **publish** runs through the tag-triggered `.github/workflows/release_napi.yml` (see [Publishing](#publishing)), and the native set is expected to eventually subsume the WASM native path. See ./crates/tsv_napi/CLAUDE.md §The npm packages.

`tsv_wasm` produces three npm packages from one crate via the `format` + `parse` cargo features (default = both): `@fuzdev/tsv_format_wasm`, `@fuzdev/tsv_parse_wasm`, and `@fuzdev/tsv_wasm` (everything + the `tsv` CLI). Each variant has its own output directory.
`tsv_wasm` produces three npm packages from one crate via the `format` + `parse` cargo features (default = both): `@fuzdev/tsv-format-wasm`, `@fuzdev/tsv-parse-wasm`, and `@fuzdev/tsv-wasm` (everything + the `tsv` CLI). Each variant has its own output directory.

```bash
# Build bindings
Expand All @@ -289,15 +289,17 @@ wasm-pack build crates/tsv_wasm --target deno --release --out-dir pkg/parse/deno

npm is the package surface (the GitHub Release per tag carries only notes + the native CLI binaries npm already ships — see the N-API paragraph below). Three packages from the WASM crate, plus the N-API set below:

- `@fuzdev/tsv_format_wasm` — format only (`--no-default-features --features format`)
- `@fuzdev/tsv_parse_wasm` — parse only; bundles hand-maintained `tsv_ast.d.ts` (`crates/tsv_wasm/types/`) + the pure-JS `no-locations` line/column reconstruction helper (`crates/tsv_wasm/npm/locations.js` + `.d.ts`)
- `@fuzdev/tsv_wasm` — full tool (both features); bundles the above and ships the `tsv` bin (`crates/tsv_wasm/npm/cli.js` — `format` + `parse` mirroring `tsv_cli`'s flags/exit codes; argv parsed by a transcription of argh's grammar, zero deps; path mode fans onto `node:worker_threads`, spawning itself as the worker and handing WASM workers the main thread's compiled module through the package's `./worker` entry — an explicit `--jobs N` is held to the native CLI's `4 × logical` ceiling (`clamp_worker_count`, restated by hand in `cli.js`), while both *defaults* are sized **per engine** and are smaller than the native CLI's: a JS pool's startup makes it worth waiting for a file-count threshold, and V8's own wasm tier-up has already claimed cores the pool would want, so the WASM copy peaks at half the physical cores where the N-API one peaks at the full count. Measured; see ./docs/cli.md §Binary Structure)
- `@fuzdev/tsv-format-wasm` — format only (`--no-default-features --features format`)
- `@fuzdev/tsv-parse-wasm` — parse only; bundles hand-maintained `tsv_ast.d.ts` (`crates/tsv_wasm/types/`) + the pure-JS `no-locations` line/column reconstruction helper (`crates/tsv_wasm/npm/locations.js` + `.d.ts`)
- `@fuzdev/tsv-wasm` — full tool (both features); bundles the above and ships the `tsv` bin (`crates/tsv_wasm/npm/cli.js` — `format` + `parse` mirroring `tsv_cli`'s flags/exit codes; argv parsed by a transcription of argh's grammar, zero deps; path mode fans onto `node:worker_threads`, spawning itself as the worker and handing WASM workers the main thread's compiled module through the package's `./worker` entry — an explicit `--jobs N` is held to the native CLI's `4 × logical` ceiling (`clamp_worker_count`, restated by hand in `cli.js`), while both *defaults* are sized **per engine** and are smaller than the native CLI's: a JS pool's startup makes it worth waiting for a file-count threshold, and V8's own wasm tier-up has already claimed cores the pool would want, so the WASM copy peaks at half the physical cores where the N-API one peaks at the full count. Measured; see ./docs/cli.md §Binary Structure)

The **N-API set** — the bare `@fuzdev/tsv` loader + five `@fuzdev/tsv-<triple>` platform packages (see ./crates/tsv_napi/CLAUDE.md §The npm packages). Each platform package ships **two binaries**: the addon (`tsv_napi.node`, `napi` profile) and the real `tsv_cli` binary (`tsv`/`tsv.exe`, plain `release` profile), and the loader's `tsv` bin is a dispatcher (`bin.js`) that execs that binary — `npx tsv` on the native set IS the native CLI (real `--jobs`, parallel discovery), the esbuild/biome shape — falling back to the shared `cli.js` JS mirror (one source with `@fuzdev/tsv_wasm`, bound to the native engine via its own `./index.js` import). The set is staged by `deno task build:napi:packages` and publishes through the tag-triggered `.github/workflows/release_napi.yml`, **never** through the single-machine `scripts/publish.ts`. The v\* tag `publish.ts` pushes triggers the 5-target matrix: container-pinned builds of both binaries (glibc **2.28 floor** on the gnu rows via almalinux:8 — Node's own binary floor; musl in rust:alpine with `-crt-static` off), each gated by the measured glibc-floor / musl-purity checks over both artifacts, per-artifact size bounds (`deno task validate:napi`, tight like `validate:artifacts`), and the npm-shape test over the real artifacts (in node:alpine for musl); then an idempotent platforms-then-loader publish (`scripts/publish_napi.ts` — refuses partial sets, re-arms the CLI binaries' executable bit that artifact transport drops; `deno task publish:napi --dry-run` runs it locally — no `--` separator, which deno task forwards literally and the script's `parseArgs` rejects — where a local run proves staging + the refusal logic and then **stops at the partial-set refusal by design**, since local staging holds only the host platform; the full-set rehearsal is the `workflow_dispatch` dry run). `workflow_dispatch` runs the whole matrix as a dry-run rehearsal by default (`dry_run=false` is the recovery path for a failed tag run — dispatched **on the tag**, since the tag↔version assertion keys on the ref and the Release job keys on it too; a `dry_run=false` dispatch off a branch is **refused** before any target builds, so a branch dispatch can only rehearse), and a weekly cron force-dry-runs it as a rot watch. Auth is a granular `NPM_TOKEN` secret from the bootstrap releases (trusted publishing is only configurable on packages that already exist on the registry); switching the workflow to npm trusted publishing (OIDC) is the pending follow-up.
**Naming.** Every npm name tsv publishes is kebab-case — `@fuzdev/tsv`, `@fuzdev/tsv-wasm`, `@fuzdev/tsv-format-wasm`, `@fuzdev/tsv-parse-wasm`, the `@fuzdev/tsv-<triple>` platform packages — one spelling with the `tsv-<triple>` release assets and the `fuzdev.tsv-format` VS Code extension (vsce forbids `_`). The Rust crates stay snake_case (`tsv_wasm`, `tsv_napi`), as does the rest of the `@fuzdev` npm scope; tsv is the one project shipping beside Rust-tooling peers whose WASM editions all spell `-wasm`, so its shipped names follow that convention rather than the ecosystem's. The crate-derived file names inside a package (`tsv_wasm.js`, `tsv_wasm_bg.wasm`, `tsv_napi.node`) are internal and follow the crate.

The **N-API set** — the bare `@fuzdev/tsv` loader + five `@fuzdev/tsv-<triple>` platform packages (see ./crates/tsv_napi/CLAUDE.md §The npm packages). Each platform package ships **two binaries**: the addon (`tsv_napi.node`, `napi` profile) and the real `tsv_cli` binary (`tsv`/`tsv.exe`, plain `release` profile), and the loader's `tsv` bin is a dispatcher (`bin.js`) that execs that binary — `npx tsv` on the native set IS the native CLI (real `--jobs`, parallel discovery), the esbuild/biome shape — falling back to the shared `cli.js` JS mirror (one source with `@fuzdev/tsv-wasm`, bound to the native engine via its own `./index.js` import). The set is staged by `deno task build:napi:packages` and publishes through the tag-triggered `.github/workflows/release_napi.yml`, **never** through the single-machine `scripts/publish.ts`. The v\* tag `publish.ts` pushes triggers the 5-target matrix: container-pinned builds of both binaries (glibc **2.28 floor** on the gnu rows via almalinux:8 — Node's own binary floor; musl in rust:alpine with `-crt-static` off), each gated by the measured glibc-floor / musl-purity checks over both artifacts, per-artifact size bounds (`deno task validate:napi`, tight like `validate:artifacts`), and the npm-shape test over the real artifacts (in node:alpine for musl); then an idempotent platforms-then-loader publish (`scripts/publish_napi.ts` — refuses partial sets, re-arms the CLI binaries' executable bit that artifact transport drops; `deno task publish:napi --dry-run` runs it locally — no `--` separator, which deno task forwards literally and the script's `parseArgs` rejects — where a local run proves staging + the refusal logic and then **stops at the partial-set refusal by design**, since local staging holds only the host platform; the full-set rehearsal is the `workflow_dispatch` dry run). `workflow_dispatch` runs the whole matrix as a dry-run rehearsal by default (`dry_run=false` is the recovery path for a failed tag run — dispatched **on the tag**, since the tag↔version assertion keys on the ref and the Release job keys on it too; a `dry_run=false` dispatch off a branch is **refused** before any target builds, so a branch dispatch can only rehearse), and a weekly cron force-dry-runs it as a rot watch. Auth is a granular `NPM_TOKEN` secret from the bootstrap releases (trusted publishing is only configurable on packages that already exist on the registry); switching the workflow to npm trusted publishing (OIDC) is the pending follow-up.

After a real publish (the tag push, or a `dry_run=false` dispatch on the tag — never the cron or a dry run), the workflow's `release` job creates the **GitHub Release** for the tag. The body is `CHANGELOG.md`'s stamped `## <version>` section (`scripts/release_notes.ts`; `deno task release:notes v<version>` previews it). The assets are every platform package's native `tsv` CLI binary (`tsv-<triple>`, `tsv-win32-x64.exe`) plus a `SHA256SUMS`, pulled back **from the registry** (`scripts/release_assets.ts`; `deno task release:assets v<version> --out <dir>`), so an asset is byte-identical to what npm serves even on a recovery re-run where the matrix rebuilds while the publish step skips versions already live; its registry reads retry for a bounded window, since the job runs seconds after the publish and a just-published version can lag the registry's read path. Every asset, the checksum file included, gets a Sigstore build provenance attestation (`actions/attest-build-provenance`, ahead of the Release so a failed attestation fails before anything is public; `gh attestation verify <file> -R fuzdev/tsv`). The first create attaches the assets in the same call, so a Release is never public without them; on a re-run an existing Release keeps its notes, the assets re-upload with `--clobber`, and the attestation is made again. The job holds the workflow's only `contents: write` and `attestations: write`, and no npm token.

A types-only `@fuzdev/tsv_ast` package is deferred — `import type` from `tsv_parse_wasm` is zero-runtime-cost; reconsider when a real consumer appears. The bare `@fuzdev/tsv` is **taken by the N-API set** above — tsv's native distribution ships under one name: the platform packages carry the `tsv_cli` binary beside the addon (un-PGO'd `release`, matching the benched artifact; PGO stays a deliberate future re-baseline).
A types-only `@fuzdev/tsv-ast` package is deferred — `import type` from `@fuzdev/tsv-parse-wasm` is zero-runtime-cost; reconsider when a real consumer appears. The bare `@fuzdev/tsv` is **taken by the N-API set** above — tsv's native distribution ships under one name: the platform packages carry the `tsv_cli` binary beside the addon (un-PGO'd `release`, matching the benched artifact; PGO stays a deliberate future re-baseline).

Version source of truth: `Cargo.toml` `[workspace.package] version` (read directly by `wasm-pack`). No root package.json, no changesets; all published packages move together.

Expand Down Expand Up @@ -838,7 +840,7 @@ cargo run -p tsv_debug metrics [--json] # line counts by crate and phase (pur

tsv ships a closed language set (TypeScript, CSS, Svelte) but is open by convention **at the Rust source/crate level**: each language crate (`tsv_ts`, `tsv_css`, `tsv_svelte`) is self-contained — owns its internal AST, parser, formatter, and convert layer — and exposes the same free-function API (`parse()`, `format()`, `convert_ast_json_bytes()`, `convert_ast_json_string()`). **No central `Language` trait, no registry, no enum dispatch.** Two properties follow:

- **Optimal artifacts**: concrete types end-to-end, no dyn dispatch; WASM tree-shakes by feature at the link level — `@fuzdev/tsv_format_wasm` excludes the convert layer, `@fuzdev/tsv_parse_wasm` the printers.
- **Optimal artifacts**: concrete types end-to-end, no dyn dispatch; WASM tree-shakes by feature at the link level — `@fuzdev/tsv-format-wasm` excludes the convert layer, `@fuzdev/tsv-parse-wasm` the printers.
- **Source-level openness**: anyone can publish a same-shaped `my_org/tsv_html_parse` crate and any downstream _Rust_ consumer can `use` it without central buy-in. Published CLI/WASM binaries still hardcode the language list (`lang_bindings!` macro), by design.

Cross-language coupling exists only where languages integrate — `tsv_svelte` depends on `tsv_ts` (for `Expression`) and `tsv_css` (for `StyleSheet`). Avoid inverting this: no central public-AST crate, no dyn `Language` trait, no workspace-level language registry. Full discussion: ./docs/architecture.md#closed-scope-open-convention.
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ tsv_discover = { path = "crates/tsv_discover" }
# Language implementations
# Default-features off ships parse + format only; consumers opt into the
# wire-JSON conversion layer with `features = ["convert"]`. The format-only
# builds leave it off (@fuzdev/tsv_format_wasm, `build:ffi:format`).
# builds leave it off (@fuzdev/tsv-format-wasm, `build:ffi:format`).
tsv_ts = { path = "crates/tsv_ts", default-features = false }
tsv_css = { path = "crates/tsv_css", default-features = false }
tsv_svelte = { path = "crates/tsv_svelte", default-features = false }
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,9 +100,9 @@ tsv ships to npm as a native package for Node and Bun, plus three WASM
packages for browsers, Deno, and platforms without a prebuilt binary:

- [`@fuzdev/tsv`](https://www.npmjs.com/package/@fuzdev/tsv) - the full tool (formatter + parser) as a prebuilt native N-API addon, shipping tsv's real native CLI binary as the `tsv` command
- [`@fuzdev/tsv_wasm`](https://www.npmjs.com/package/@fuzdev/tsv_wasm) - the full tool with the same `tsv` CLI, as WASM
- [`@fuzdev/tsv_format_wasm`](https://www.npmjs.com/package/@fuzdev/tsv_format_wasm) - formatter only (smaller)
- [`@fuzdev/tsv_parse_wasm`](https://www.npmjs.com/package/@fuzdev/tsv_parse_wasm) - parser + JSON AST only (smallest)
- [`@fuzdev/tsv-wasm`](https://www.npmjs.com/package/@fuzdev/tsv-wasm) - the full tool with the same `tsv` CLI, as WASM
- [`@fuzdev/tsv-format-wasm`](https://www.npmjs.com/package/@fuzdev/tsv-format-wasm) - formatter only (smaller)
- [`@fuzdev/tsv-parse-wasm`](https://www.npmjs.com/package/@fuzdev/tsv-parse-wasm) - parser + JSON AST only (smallest)

```bash
npm i @fuzdev/tsv
Expand All @@ -120,7 +120,7 @@ import {parse_svelte, type Root} from '@fuzdev/tsv';
const ast: Root = parse_svelte('<script>const x = 1;</script>');
```

`@fuzdev/tsv` and `@fuzdev/tsv_wasm` are drop-in swaps.
`@fuzdev/tsv` and `@fuzdev/tsv-wasm` are drop-in swaps.
The native package is prebuilt
for Linux (x64 gnu and musl, arm64 gnu), macOS arm64, and Windows x64.
As with other wasm packages, browsers must call `await init()`.
Expand Down
4 changes: 2 additions & 2 deletions benches/js/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -923,7 +923,7 @@ run's output.
out of `init_implementations` (`init_required`) instead of joining `unavailable`,
and their slots are correspondingly non-`undefined` in `ImplementationSet` — a
broken tree, not a machine coming up short. Before that, a wasm bundle that was
present but wouldn't load published a report with every `tsv_wasm-*` row silently
present but wouldn't load published a report with every `tsv-wasm-*` row silently
gone behind one ⚠ line, and five diagnostics each hand-rolled their own
`if (!impls.native) throw`. Note the division of labour with the freshness guard:
`check_artifact_freshness` makes a MISSING artifact fatal, a present-yet-unloadable
Expand Down Expand Up @@ -1469,7 +1469,7 @@ internal state), the coverage report and skip counts make it visible without
(`lib/oxc_wasm.ts` does; `lib/oxc.ts` mirrors the form defensively). Two guards
exist: the single-read pattern at the wrappers, and `bench.ts`'s
`check_variant_parity` — after pre-flight, same-engine pairs
(tsv↔tsv_wasm variants, oxc-parser↔oxc-parser-wasm, yuku-parser↔yuku-parser-wasm,
(tsv↔tsv-wasm variants, oxc-parser↔oxc-parser-wasm, yuku-parser↔yuku-parser-wasm,
rsvelte-parse↔rsvelte-parse-skip-expr-loc) are compared file-for-file and
any accept-set divergence prints a `⚠ variant parity` warning (same engine ⇒ a
divergence is a binding-boundary bug, not an engine difference).
Expand Down
Loading
Loading