diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml new file mode 100644 index 0000000..981a301 --- /dev/null +++ b/.github/workflows/pages.yml @@ -0,0 +1,61 @@ +name: docs site + +# Publishes docs/ to GitHub Pages. The site is plain HTML with inline CSS and +# no external requests, so there is nothing to build — the job uploads the +# directory as it stands in the repository. +# +# This does nothing until Pages is switched on once, by hand: +# Settings -> Pages -> Build and deployment -> Source: GitHub Actions +# Until then the deploy step fails with "Pages is not enabled", which is the +# intended state for a repository that has not decided to publish yet. + +on: + push: + branches: [main] + paths: + - 'docs/**' + - '.github/workflows/pages.yml' + workflow_dispatch: + +permissions: + contents: read + pages: write + id-token: write + +# One deploy at a time, and never cancel one that is already running: a +# half-published site is worse than a slightly stale one. +concurrency: + group: pages + cancel-in-progress: false + +jobs: + deploy: + runs-on: ubuntu-latest + timeout-minutes: 10 + environment: + name: github-pages + url: ${{ steps.deploy.outputs.page_url }} + + steps: + - uses: actions/checkout@v4 + + - name: Check the site loads nothing from the network + # Outbound links are fine and expected. A remote *asset* is + # not: a CDN script, stylesheet, font or image makes the page depend on + # someone else's uptime, and it fails quietly for whoever has it cached. + # Cheaper to fail here than to find out from a reader. + run: | + if grep -rInE '(]+src=|]+href=|]+src=|@import|url\()["'"'"']?https?://' \ + docs --include='*.html' --include='*.css' --include='*.svg'; then + echo "::error::docs/ loads an asset from the network. Inline it instead." + exit 1 + fi + + - uses: actions/configure-pages@v5 + + - uses: actions/upload-pages-artifact@v3 + with: + path: docs + + - id: deploy + uses: actions/deploy-pages@v5 diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..0937385 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,134 @@ +# AGENTS.md + +Orientation for an agent landing in this repository cold. `CLAUDE.md` is a +symlink to this file — edit this one. + +## What this is + +`webcodecs-census` finds leaked WebCodecs objects in a browser app and hands +back the line of code that allocated them. WebCodecs objects hold resources +from a finite pool outside the JS heap; GC never reclaims them, only `close()` +does, and nothing in the platform tells you that you leaked one. The hard part +is that decoders live in Web Workers, which page-level monkey-patching cannot +reach — so the exact path drives Chrome over the DevTools Protocol and injects +into each worker at a `beforeScriptExecution` pause, before its first line. + +Public repository, published to npm as three packages under `@motionvector/`. + +| Path | What | +| --- | --- | +| `packages/core` | The instrumentation and the assertion API. No dependencies. Also builds the injectable IIFE shim. | +| `packages/cdp` | Injects the shim into a running Chrome — page, iframes, workers — over CDP. | +| `packages/mcp` | An MCP server wrapping the above, so an agent can use it. | +| `extension/` | A Chrome MV3 extension. Patch mode (no debugger) and exact mode (`chrome.debugger`). | +| `docs/` | The documentation site. Plain HTML, no build step. See `docs/README.md`. | +| `test/` | Real-browser tests. No mocks. | +| `scripts/` | Release plumbing: `version.mjs`, `preflight.mjs`, `publish.mjs`, `release-notes.mjs`, `make-diagram.mjs`. | + +## The one rule that matters + +**A change must not let this tool report "no leaks" for an app that is +leaking.** Everything else is style. A leak detector that silently sees nothing +is worse than no leak detector, because it converts an open question into a +wrong answer. `CONTRIBUTING.md` has the long version. + +Several things exist only to protect that property, and they are not +refactoring targets: + +- `installCensus()` wraps each patch step separately and records failures in + `problems[]` instead of throwing. +- Patch mode reports every worker it could not wrap, with the reason. +- `checkLeaks()` never lets `types` filter away `collectedUnclosed`, and never + prints an unqualified all-clear while an unenforced type holds live objects. +- `test/platform-assumptions.test.mjs` asserts the undocumented Chrome + behaviour injection depends on, so a browser change is reported as a browser + change. + +The corollary matters too: the tool must not invent leaks either. A codec the +platform closed after an error is not a leak, and reporting it as one would be +the same class of failure pointed the other way. + +If you add a path that can silently observe less than it appears to, add the +counter or the `problems[]` entry that makes it visible. + +## Build and test + +```bash +npm install +npm run build:all # core + cdp + mcp, then the extension, then test fixtures +npm test # node --test over test/*.test.mjs +npm run typecheck # tsc over all three packages; needs npm run build first +``` + +Tests drive a **real Chrome** — there are no mocks, deliberately. A mocked +`VideoDecoder` would have hidden every bug worth finding here. They find a +Chrome for Testing in the Puppeteer cache, or you point at one: + +```bash +CHROME_PATH="$HOME/.cache/puppeteer/chrome/mac_arm-151.0.7922.71/chrome-mac-arm64/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing" npm test +``` + +Never the user's own Chrome. Instrumenting that would touch their profile and +their session. + +### The git-worktree trap + +**Run `npm ci` inside the worktree before trusting a `test` or `typecheck` +result.** A worktree with no `node_modules` of its own resolves +`@motionvector/*` upwards to the main checkout, so: + +- `typecheck` checks `mcp` against whatever declarations are built over there — + reporting errors against code that is fine, or missing errors in code that is + not. +- The tests import their entry points by relative path, but `packages/cdp` + loads the shim by package name, so a run in a bare worktree tests the *main + checkout's* shim, not yours. + +`ls -la node_modules/@motionvector` should show symlinks pointing back into the +worktree you are standing in. + +## Releasing + +Maintainers only, and all three packages move together — `-cdp` and `-mcp` +depend on an **exact** version of the core, so bumping one alone publishes a +package whose dependency does not exist. + +```bash +node scripts/version.mjs minor # or patch / major / an explicit version +git commit -am "release: v0.3.0" && git tag v0.3.0 +git push origin main --tags +``` + +`scripts/version.mjs` moves the three `package.json` versions, rewrites the +cross-dependencies, rewrites the two version strings that live in source +(`VERSION` in `packages/core/src/census.ts` and the name the MCP server +announces in `packages/mcp/src/index.ts`), and promotes the `Unreleased` +section of `CHANGELOG.md`. It refuses a dirty tree, and it fails loudly rather +than silently if a version pattern stops matching — those two stamps shipped +two releases stale once because nothing rewrote them. + +The `v*` tag triggers `.github/workflows/release.yml`, which refuses a tag that +disagrees with the packages, runs the full browser suite against pinned Chrome +*before* publishing, then publishes all three in dependency order over OIDC +trusted publishing with provenance, and opens a GitHub Release from that +version's changelog section with the packed extension attached. There is no npm +token. + +## Verifying, not asserting + +Claims in this repository are measured. The overhead figures, the globals +table, the comparison against heap snapshots — each came from a run, not from +reasoning about what ought to happen. Hold new claims to that. If you cannot +measure it, write down that you could not, or leave it out. + +## Style + +Short sentences, active voice, no filler. Comments explain *why*, particularly +where the reason is a platform quirk that will look like a mistake to the next +reader. Do not add comments that restate the code. + +## Publishing anything public + +This is a public repository. Do not push, open a pull request, publish to npm, +or enable GitHub Pages without the maintainer saying so in the conversation. +Draft it, then hand it over. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 0000000..47dc3e3 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/README.md b/README.md index 92bf32c..c34e363 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,7 @@ **Find leaked `VideoFrame`s, `AudioData` and codecs in a WebCodecs app — including inside Web Workers — and get the line of code that allocated them.** +[**Documentation**](https://motionvector-dev.github.io/webcodecs-census/)  ·  [Why it's hard](#why-this-is-hard-and-why-it-didnt-exist)  ·  [Use from an agent](#use-it-from-an-agent)  ·  [Use in CI](#use-it-in-ci)  ·  @@ -160,7 +161,7 @@ import { expectNoLeakedFrames } from '@motionvector/webcodecs-census'; test('the editor releases every frame it decodes', async () => { await playThroughTimeline(); - expectNoLeakedFrames(await session.census(), { minAgeMs: 1000 }); + expectNoLeakedFrames(await session.census(), { allow: { VideoFrame: 2 } }); }); ``` @@ -249,6 +250,14 @@ The core makes no network requests of any kind and has no runtime dependencies. - Patch mode changes `self.location` inside wrapped workers to the loader blob URL. Workers using `import.meta.url` are unaffected; workers building paths from `self.location` are not. - Exact mode cannot share a tab with an open DevTools window. Chrome allows one debugger client. +## Documentation + +The full reference lives at +**[motionvector-dev.github.io/webcodecs-census](https://motionvector-dev.github.io/webcodecs-census/)** +— the assertion API field by field, the CDP driver, the MCP server, the +extension, CI recipes, and the limits above in more detail. Its source is +[`docs/`](./docs), which is plain HTML with no build step. + ## Development ```bash diff --git a/docs/.nojekyll b/docs/.nojekyll new file mode 100644 index 0000000..e69de29 diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..d7f20a4 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,48 @@ +# docs/ + +The documentation site, and the diagram the root README uses. + +| Path | What | +| --- | --- | +| `index.html` | What the tool is, the problem, and why the worker case is hard | +| `quickstart.html` | Install and a first leak report, three ways | +| `api.html` | The core and assertion API, field by field | +| `cdp.html` | The CDP driver | +| `mcp.html` | The MCP server | +| `extension.html` | The browser extension, patch mode and exact mode | +| `ci.html` | CI recipes | +| `limits.html` | What it cannot see | +| `injection-*.svg` | The two-phase injection diagram, one file per theme. Generated by `npm run build:diagram`; also embedded in the root README | +| `.nojekyll` | Serve the directory as-is rather than through Jekyll | + +## Publishing + +`.github/workflows/pages.yml` uploads this directory unchanged. Nothing is +built, so what is committed is exactly what is served — open any page with +`file://` and it works. + +Pages has to be switched on once by hand, under **Settings → Pages → Source: +GitHub Actions**. Until then the deploy step fails, which is the right state +for a repository that has not decided to publish. + +## Editing + +Two rules, both of which the workflow enforces or the pages depend on. + +**Nothing may be loaded from the network.** No CDN scripts, no remote fonts, no +external stylesheets. Outbound `` links are fine; a remote *asset* is +not, because it makes the page depend on someone else's uptime and fails +quietly for anyone who has it cached. The deploy workflow greps for this and +fails the build. + +**Every page carries the same ` + + +
+
+ WebCodecs Census docs + +
+ + + +
+
+
+ +
+
+

API reference

+

The assertion API

+

+ Everything exported by @motionvector/webcodecs-census. Two halves: the + instrumentation that runs inside a context, and the verdict layer that turns a census into + pass or fail. +

+ +
import {
+  installCensus, localCensus, timeline, resetCensus, VERSION,
+  checkLeaks, expectNoLeaks, expectNoLeakedFrames, summarize, totalLive,
+  TRACKED,
+} from '@motionvector/webcodecs-census';
+ +

Verdicts

+ +

checkLeaks(censuses, options?)

+ +

+ Builds a LeakReport without throwing. checkLeaks(...).ok is the + boolean form of every assertion below. +

+ +
const report = checkLeaks(await session.census(), { types: 'all', allow: { VideoFrame: 2 } });
+if (!report.ok) console.error(report.message);
+ +

LeakOptions

+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
OptionTypeDefaultWhat it does
typesTrackedType[] | 'all'['VideoFrame', 'AudioData', 'ImageBitmap'] + Which types count as live too long. The default is the frame-like types, + because a long-lived decoder is normal and a long-lived frame almost never is. + 'all' resolves to every entry in TRACKED. +
allowPartial<Record<TrackedType, number>>{} + Tolerated live count per type, summed across every context. A steady-state pipeline + legitimately holds a few. Applies to live counts only — never to + collectedUnclosed. +
minAgeMsnumber0 + Ignores live objects younger than this — a decode in flight is not a leak. Decides + the verdict, not just the printed attribution (v0.3.1; before that it filtered + report.sites only). +
failOnOverClose (v0.3.0)booleanfalse + Fail when a close() threw. Off by default: the usual cause is a library + closing a codec twice, which the app that would see the failure cannot fix. Turn it + on for code you own. +
+
+ +
+ How minAgeMs stays exact +

+ The census carries liveAges: the age of every live object, per type, oldest + first, capped at liveAgesCap (256). Keeping the oldest is what makes + the count exact — anything dropped is younger than the youngest age kept, so it cannot + clear a threshold the kept ages already fall below. +

+

+ The one exception is saturation, where every kept age clears the bar. There the count is + an honest lower bound — at least 256 VideoFrame still live … 9000 live in + total — with the exact total in report.liveBounded. The verdict is the + same either way; only the claim changes. +

+

+ A census taken by a shim older than v0.3.1 carries no ages. Rather than ignore the option + silently, the report leaves the counts unfiltered — never fewer than the truth — sets + minAgeMsApplied to false, and names the contexts it could not + filter. +

+
+ +

LeakReport

+ +
+ + + + + + + + + + + + + + +
FieldTypeMeaning
okbooleanThe verdict. See below for exactly what it is computed from.
livePartial<Record<TrackedType, number>>Live objects of the enforced types, summed across contexts. Age-filtered when minAgeMs asked for it.
liveBounded (v0.3.1)Partial<Record<TrackedType, number>>Types where live is "at least this many" because the age filter saturated the census cap. The value is the exact total live count.
minAgeMsApplied (v0.3.1)booleanWhether minAgeMs reached the verdict. False when it was not asked for, or when a census carries no ages.
unenforcedLivePartial<Record<TrackedType, number>>Live objects of the types types left out. Reported, never failed on.
collectedUnclosedPartial<Record<TrackedType, number>>GC'd without close(), across every tracked type. types cannot filter this away.
enforcedTrackedType[]What types resolved to.
sites(LeakSite & { context: string })[]Allocation sites holding live objects of the enforced types, worst first.
overCloses (v0.3.0)(OverCloseSite & { context: string })[]close() calls that threw, worst first. Always reported; only decides ok under failOnOverClose.
messagestringThe human-readable form. This is what expectNoLeaks throws.
+
+ +

How ok is decided

+ +

ok is true when all of these hold:

+ + +
+ Two invariants the filter cannot break +

+ A definitive leak is never filtered out. An object the GC collected while + it was still open failed to release a resource for its entire lifetime. That fails the + check whatever types says. A filter aimed at live frames must not hide a + decoder that was dropped on the floor. +

+

+ An unchecked type is never reported clean. If a type you left out still + holds live objects, the message names it rather than printing an unqualified all-clear: +

+
No leaks in VideoFrame, AudioData, ImageBitmap — but VideoDecoder=47 still
+live and not enforced. Pass types: 'all' to check those too.
+
+ +

What the platform does behind your back (v0.3.0)

+ +

+ Two things happen to a codec that no close() call in your source explains. + Getting either one wrong makes the report lie in a different direction. +

+ +

A failing codec is closed by the platform

+ +

+ The spec's Close algorithm sets [[state]] to "closed" + before it invokes your error callback, so no close() call ever reaches + the instrumentation. Left alone, that codec stays counted live and, once collected, is filed + as collectedUnclosed — "definitively leaked" for a resource the platform had + already reclaimed. +

+

+ The census reconciles live codecs against their own state, records the + departure as the closedByPlatform fate, and does not call it a leak. + Reconciliation runs on every sample tick and at the top of + localCensus(), so a census taken straight after a decode error is already + correct. A detector that invents leaks in an error-heavy pipeline is worse than one that + stays quiet. +

+ +

Closing an already-closed codec throws

+ +

+ Measured on Chrome 151: the four codec types throw InvalidStateError; the three + frame types are idempotent and throw nothing. From a library closing out of a floating + .finally(), that surfaces as an unhandled rejection no application code can + catch — the platform reports it to nobody who can act on it. +

+

+ Those calls are counted, grouped by type, message and calling line, in + ContextCensus.overCloses and LeakReport.overCloses. The throw is + observed, never swallowed: the caller still sees what the platform threw, or instrumenting + the app would change how it behaves. +

+ +
3 close() call(s) threw — a codec was closed twice:
+  3x VideoDecoder: Cannot call 'close' on a closed codec (in worker)
+      at decodePump (pipeline.js:812:20)
+ +

+ An over-close is a lifecycle defect, not a leak, so it never fails a check on its own. Pass + failOnOverClose: true for code you own. +

+ +
+ + + + + + + + +
OverCloseSiteMeaning
typeWhich tracked type the call was made on.
messageWhat the platform threw, e.g. Cannot call 'close' on a closed codec.
stackThe line that called close().
countHow many times that exact site threw. Grouped because the thing doing it usually runs per frame.
+
+ +

expectNoLeaks(censuses, options?)

+ +

+ Runs checkLeaks and throws new Error(report.message) unless + ok. The message carries the counts and up to five allocation sites with three + stack frames each — enough to act on from a CI log. +

+ +
expectNoLeaks(await session.census(), { types: 'all' });
+ +

expectNoLeakedFrames(censuses, options?)

+ +

+ The common case, named for what it means: expectNoLeaks with + types: ['VideoFrame']. Note that options is spread + after the default, so passing your own types overrides it. +

+ +
expectNoLeakedFrames(await session.census());
+expectNoLeakedFrames(await session.census(), { allow: { VideoFrame: 3 } });
+ +

+ Because only VideoFrame is enforced, a live AudioData shows up in + unenforcedLive and in the message rather than failing the test. +

+ +

summarize(censuses)

+ +

+ A compact digest, sized for an agent or a CI log to read in one go. One line per context + plus the default verdict. Deliberately small: a full census is mostly stack strings. +

+ +
2 context(s): main, worker
+  main (3s): nothing live
+  worker (3s): VideoDecoder=1 VideoFrame=5 | media 4 (1 stalled)
+
+5 VideoFrame still live (allowed 0).
+
+Held by:
+  5x VideoFrame (decoded, oldest 100ms) in worker
+      at VideoSample.toVideoFrame (pipeline.js:17261:14)
+      (frame emitted by this VideoDecoder)
+ +

+ summarize uses the default types. If you are enforcing + the codecs, call checkLeaks yourself and print its message. +

+ +

totalLive(censuses, type)

+ +

Sums one type's live count across contexts. Returns a number.

+ +
const frames = totalLive(await session.census(), 'VideoFrame');
+ +

TRACKED

+ +

+ The seven types whose resources GC cannot reclaim, in order. TrackedType is + the union of its members. +

+ +
['VideoDecoder', 'VideoEncoder', 'AudioDecoder', 'AudioEncoder',
+ 'VideoFrame', 'AudioData', 'ImageBitmap']
+ +

Instrumentation

+ +

installCensus(options?)

+ +

+ Patches this context's globals. Safe to call more than once — a second call only updates + context and returns, so re-running it after HMR or a second bundle chunk does + not reset the counters or double-patch a constructor. +

+ +
+ + + + + + + + + +
OptionTypeDefaultWhat it does
contextstringguessedName for this context in reports. The guess is main, worker, shared-worker, service-worker or unknown.
sampleIntervalMsnumber | false500Rolling timeline interval. false disables sampling.
keepSamplesnumber240How many samples to retain. Older ones are dropped.
stackDepthnumber8Frames of allocation stack to keep per object.
warnOnCollectbooleantrueLog a console warning when an object is GC'd without close().
+
+ +

+ Each patch step is wrapped separately. A global that is missing in this context costs you + that one patch and is recorded in problems[] — it never aborts the install, and + it is never swallowed. +

+ +

+ installCensus also exposes globalThis.__webcodecsCensus: a + function returning the local census, with .local(), .timeline(), + .reset() and .version attached. That is how the CDP driver and + the extension read a context they injected into. +

+ +

localCensus()

+ +

+ A ContextCensus for this context only. Collecting a worker's + census means running this inside that worker. +

+ +
+ + + + + + + + + + + + + + + + + + + +
FieldTypeMeaning
contextstringThe context name.
uptimeMsnumberHow long the census has been installed. Makes rates computable.
enteredRecord<string, number>Keyed Type:origin, e.g. VideoFrame:decoded.
leftRecord<string, number>Keyed Type:fate — closed, transferred, or closedByPlatform (v0.3.0).
livePartial<Record<TrackedType, number>>Open right now, by type.
liveAges (v0.3.1)Partial<Record<TrackedType, number[]>>Ages in ms of the live objects, per type, oldest first and capped at liveAgesCap.
liveAgesCap (v0.3.1)numberThe cap liveAges was truncated at, carried so a saturated record says what it is.
collectedUnclosedPartial<Record<TrackedType, number>>Collected by GC without close(). Unambiguous.
closedUnseennumberClosed here but never seen entering — usually a receive path the message scanner missed.
overCloses (v0.3.0)OverCloseSite[]close() calls that threw, worst first. A lifecycle bug, not a leak.
leakSitesLeakSite[]Live objects grouped by allocation site, worst first.
oldestLiveLiveObject[]The ten oldest live objects, each with its stack and age.
mediaElementsMediaElementCensus{ total, stalled, byReadyState }.
timelineSample[]The rolling samples.
problemsstring[]Anything that could not be instrumented here. Non-empty means the numbers are a floor, not a total.
+
+ +

timeline()

+ +

The rolling samples on their own. Each Sample:

+ +
+ + + + + + + + + +
FieldMeaning
tms since install.
liveLive counts by type at that moment.
gained / lostEntered and left since the previous sample, by type.
activity{ decodeCalls, encodeCalls, outputs, errors, queued, configured }. queued is summed decodeQueueSize/encodeQueueSize across live codecs — backpressure.
mediaElementsElement totals and how many are stalled.
+
+ +

+ This is what separates a busy pipeline from a wedged one. Decoder idle, queue empty and the + live count frozen means wedged; a snapshot alone reads it as normal. +

+ +
+ Timers arrive late in a worker +

+ A worker paused before its first line has VideoFrame but not + setInterval. The sampler therefore starts on the first tracked allocation + after the worker resumes, not at install time — so an early census can legitimately have + an empty timeline. +

+
+ +

resetCensus()

+ +

+ Clears the counters without unpatching. Tests need this; nothing else should. Objects that + were live before the reset are forgotten, so a reset mid-run hides a real leak. +

+ +

VERSION

+ +

+ The core's version string, stamped into every census payload and readable in-page as + __webcodecsCensus.version. The release script rewrites it, and fails loudly + if it ever stops matching its pattern. +

+ +

Subpath exports

+ +
+ + + + + + + +
SpecifierWhat it gives you
@motionvector/webcodecs-censusThe ESM API above.
@motionvector/webcodecs-census/shimSHIM_SOURCE: the census bundled as a self-installing IIFE, as a string, for injecting into a context you do not control.
@motionvector/webcodecs-census/shim.txtThe same IIFE as a file, for serving or reading directly.
+
+ +

+ The injected build reads its options from globalThis.__webcodecsCensusOptions + before installing, because an IIFE has no callable export. + The CDP driver does this for you. +

+
+
+ + + + + + diff --git a/docs/cdp.html b/docs/cdp.html new file mode 100644 index 0000000..47589d0 --- /dev/null +++ b/docs/cdp.html @@ -0,0 +1,506 @@ + + + + + +CDP driver — WebCodecs Census + + + + + + + +
+
+ WebCodecs Census docs + +
+ + + +
+
+
+ +
+
+

CDP driver

+

@motionvector/webcodecs-census-cdp

+

+ Instrument a page and every one of its Web Workers from outside the app, with no change to + the code being measured — and get in before a worker's first line runs. +

+ +
npm install --save-dev @motionvector/webcodecs-census-cdp
+ +
import { attach, launchChrome } from '@motionvector/webcodecs-census-cdp';
+import { summarize } from '@motionvector/webcodecs-census';
+
+const chrome = await launchChrome({ executablePath: CHROME });
+const session = await attach({ browserURL: chrome.browserURL });
+
+await session.navigate('http://localhost:5173/');
+// …drive the app…
+
+console.log(summarize(await session.census()));
+
+session.detach();
+await chrome.kill();
+ +

attach(options)

+ +

Returns a CensusSession. Pass either a browser endpoint or a page socket.

+ +
+ + + + + + + + + + + +
OptionTypeWhat it does
browserURLstringDevTools HTTP endpoint, e.g. http://127.0.0.1:9222. The driver resolves a page target from it.
webSocketDebuggerUrlstringOr a page target's WebSocket URL directly.
matchUrlstring | RegExpChoose among page targets when several are open. A string matches as a substring.
install{ sampleIntervalMs?, keepSamples?, stackDepth? }Handed to installCensus inside each context.
shimSourcestringOverride the injected source. Defaults to the built census shim.
onContext(info) => voidCalled per instrumented context with { type, url, sessionId }.
onError(e: Error) => voidCalled when a context could not be instrumented. Injection failures are reported here rather than thrown.
+
+ +

+ Ordering inside attach is load-bearing: auto-attach is armed and the document + script registered before anything navigates, or a worker can start unobserved. + The shim is also evaluated once against the current document, so attaching to an + already-loaded page still instruments it — installing twice is a no-op by design. +

+ +

CensusSession

+ +
+ + + + + + + + + + + + + +
MethodWhat it does
census() + Snapshot every context. Each is queried on its own CDP session, so no cross-context + message channel is needed and a wedged worker cannot stop the others reporting. + Each entry carries targetUrl and targetType alongside the + ContextCensus fields. +
contexts()Every context currently instrumented, as { sessionId, type, url }. The page's own session has sessionId: null.
evaluate(expression, sessionId?)Run an expression in one context, or the page by default. Awaits promises and returns the value, or null if the call could not be made.
navigate(url)Page.navigate. The document script is already registered, so the new document is instrumented from its first line.
redirect(rules)Serve matching URLs from somewhere else — see below.
detach()Stop listening and close the socket. Does not close the browser.
+
+ +

redirect(rules)

+ +

+ Pins a request to a different URL, so a large media asset can be served from a local copy + and a run is fast and repeatable without editing the app under test. +

+ +
await session.redirect([
+  { from: 'cdn.example.com/big.mp4', to: 'http://127.0.0.1:8081/media.mp4' },
+  { from: /\/segments\/.*\.m4s$/, to: 'http://127.0.0.1:8081/fixture.m4s' },
+]);
+ +

+ A string rule matches as a substring; a RegExp is tested against the full URL. + Everything else continues untouched. +

+ +

launchChrome(options)

+ +

+ Launches Chrome with a throwaway profile. It never reuses, and never kills, a browser you + already have open. +

+ +
+ + + + + + + + + +
OptionTypeDefaultWhat it does
executablePathstringrequiredPath to a Chrome or Chromium binary.
portnumber0Debugging port. 0 lets Chrome choose a free one, which is the safe default — a fixed port collides with any browser you already have open for debugging.
headlessbooleantruePass false to see the window.
argsstring[][]Extra flags. --user-data-dir and the debugging port are always the driver's.
startupTimeoutMsnumber20000How long to wait for the debugging port.
+
+ +

+ Returns { process, browserURL, kill }. kill() signals only the + process it spawned and removes only the profile it made, waiting for Chrome's helper + processes to exit first — removing the profile straight away races them and fails with + ENOTEMPTY. +

+ +
+ Why a throwaway profile, always +

+ Attaching to a browser you are signed into risks touching your session, and a shared + profile makes runs non-repeatable. Chrome's DevTools Protocol is also + unauthenticated by design: anything that can reach the port controls the + browser. Never expose a debugging port beyond localhost. +

+
+ +

Also exported

+ +
+ + + + + +
findPageTarget(origin, match?)Resolve a page target from a DevTools endpoint. Throws with the targets it did see, which is usually the fastest way to find out you pointed at the wrong browser.
CdpClientA minimal flat-session CDP client, if you need one. It runs on the WebSocket built into Node and has no dependencies — anything larger would pull in a browser-automation stack.
+
+ +

How it reaches a worker

+ +

Two Chrome behaviours make a worker reachable, and the second one has a trap in it.

+ + + +

+ At that auto-attach pause a dedicated worker's global is only half built. Measured on + Chrome 151: +

+ +
+ + + + + + + +
At the auto-attach pause
VideoFrame, AudioData, ImageBitmap, EncodedVideoChunkpresent
VideoDecoder, VideoEncoder, AudioDecoder, AudioEncoderabsent
setInterval, setTimeout, queueMicrotaskabsent
+
+ +

+ Patch there and you instrument the frame types but miss every codec. So the driver arms a + beforeScriptExecution instrumentation breakpoint and resumes into it. That + second pause has the global fully populated and is still before the worker's own script + runs. The breakpoint is removed immediately after injection — leaving it armed would pause + on every subsequent script the worker loads and stall the app under test. +

+ +

+ If the Debugger domain is unavailable in a target, the driver falls back to + injecting at the earlier pause: weaker, but it still catches frame allocations. A worker is + always resumed, even when injection failed — leaving one paused would hang + the page, which is far worse than a gap in the census. +

+ +

+ Auto-attach is not recursive, so each attached target arms it again for its own children. + That is what reaches nested workers. data: and blob: URL workers + are covered too, and iframes get the document script rather than the pause dance. +

+ +
+ Sequence diagram of the two-phase injection into a Web Worker. + Sequence diagram of the two-phase injection into a Web Worker. +
+ +

+ This behaviour is measured rather than specified, so the repository asserts it directly in + test/platform-assumptions.test.mjs and prints what it found at each pause. A + Chrome change is then reported as a Chrome change, rather than surfacing as a mysterious + failure somewhere else. +

+
+
+ + + + + + diff --git a/docs/ci.html b/docs/ci.html new file mode 100644 index 0000000..479781f --- /dev/null +++ b/docs/ci.html @@ -0,0 +1,506 @@ + + + + + +CI recipes — WebCodecs Census + + + + + + + +
+
+ WebCodecs Census docs + +
+ + + +
+
+
+ +
+
+

CI recipes

+

Make a leak a test failure

+

+ A leaked frame is easy to ship and hard to notice. Asserted in CI, it becomes a red build + on the pull request that introduced it instead of a mystery six months later. +

+ +

The shape of it

+ +
    +
  1. Install a pinned Chrome for Testing on the runner.
    2. +
  2. Serve the app, launch Chrome, attach the census.
    3. +
  3. Drive the workload you care about.
    4. +
  4. Assert. Fail with the allocation site in the log.
    5. +
+ +

+ Pin the browser version. The injection path depends on undocumented Chrome behaviour, so an + unpinned browser turns a Chrome release into a mysterious failure on an unrelated pull + request. This repository pins one version for its blocking job and runs a separate, + non-blocking job against stable on a schedule — a browser change is then news, not a + blocker. +

+ +

GitHub Actions

+ +
name: leak check
+
+on: [push, pull_request]
+
+jobs:
+  leaks:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: npm
+
+      - run: npm ci
+
+      - name: Install Chrome for Testing
+        run: |
+          # Prints "chrome@<version> <path>"; the path is the last field.
+          OUT=$(npx --yes @puppeteer/browsers install chrome@151.0.7922.71)
+          echo "$OUT"
+          echo "CHROME_PATH=${OUT##* }" >> "$GITHUB_ENV"
+
+      - run: npm run build
+      - run: node --test --test-timeout=120000 test/leaks.test.mjs
+ +

+ Use Node 22. The CDP client uses Node's built-in global WebSocket and imports + no WebSocket library. +

+ +

The test

+ +
import { test, after } from 'node:test';
+import { attach, launchChrome } from '@motionvector/webcodecs-census-cdp';
+import { expectNoLeaks, summarize } from '@motionvector/webcodecs-census';
+
+const chrome = await launchChrome({
+  executablePath: process.env.CHROME_PATH,
+  // Chrome's sandbox needs privileges a CI container usually will not grant,
+  // and the failure is an opaque early exit rather than a message about it.
+  args: process.env.CI ? ['--no-sandbox', '--disable-dev-shm-usage'] : [],
+});
+
+const session = await attach({ browserURL: chrome.browserURL });
+
+after(async () => {
+  session.detach();
+  await chrome.kill();
+});
+
+test('the editor releases every frame it decodes', async () => {
+  await session.navigate('http://127.0.0.1:5173/');
+  await session.evaluate('window.playThroughTimeline()');
+
+  const censuses = await session.census();
+  console.log(summarize(censuses));          // useful even when it passes
+  expectNoLeaks(censuses, { types: 'all' });
+});
+ +

+ Print the summary whether or not it fails. A passing run that quietly drifts from 2 live + frames to 200 is the thing you want to catch before it crosses a threshold. +

+ +
+ Watch the problems array +

+ census.problems lists anything a context could not instrument. Non-empty + means the counts below it are a floor, not a total. Assert on it too, or a build that + instrumented nothing passes for the same reason an app with no leaks does. +

+
for (const c of censuses) assert.deepEqual(c.problems, [], `${c.context} was not fully instrumented`);
+
+ +

Without a driver

+ +

+ If your suite already runs in a browser — a Karma, Vitest browser-mode or Playwright + component test — install the core directly and assert on the local census. Remember it + covers only the context it runs in, so a worker's census has to be collected inside that + worker. +

+ +
import { installCensus, localCensus, expectNoLeakedFrames } from '@motionvector/webcodecs-census';
+
+installCensus({ context: 'test' });
+
+test('decoding a clip leaks nothing', async () => {
+  await decodeClip();
+  expectNoLeakedFrames([localCensus()]);
+});
+ +

Tuning the threshold

+ +

A steady-state pipeline legitimately holds a few objects open. Say so explicitly.

+ +
+ + + + + + + + + + + + + + + + + + + + + + + + +
WantDo
Frames only, which is the usual first gateexpectNoLeakedFrames(censuses)
Hold the codecs to the same standardexpectNoLeaks(censuses, { types: 'all' })
Tolerate a small steady-state poolexpectNoLeaks(censuses, { allow: { VideoFrame: 4 } })
Fail when something closes a codec twice (v0.3.0)expectNoLeaks(censuses, { failOnOverClose: true })
Report rather than failconst { ok, message } = checkLeaks(censuses)
+
+ +

+ Whatever you pass, an object the GC collected while it was still open fails the check, and + a type left out of types is named in the message rather than quietly reported + clean. Both are deliberate — see the API reference. +

+ +

Reading a failure

+ +
Error: 3 VideoFrame garbage collected without close() — definitively leaked.
+58 VideoFrame still live (allowed 0).
+Not enforced, and still live: VideoDecoder=1.
+
+Held by:
+  58x VideoFrame (decoded, oldest 124609ms) in worker
+      at PackagerWorker.setupDecoder (worker.js:1756:21)
+      (frame emitted by this VideoDecoder)
+ +
+ + + + + + + + + + + + + + + + + + + + +
LineRead it as
garbage collected without close()Certain. The resource was held for the object's whole lifetime. Fix this first.
still live (allowed 0)Open at census time. Check the age — oldest 124609ms is not a pipeline mid-flight.
decodedThe platform made this frame and handed it to a decoder's output callback. The stack is that decoder's construction site, which is the line to look at.
Not enforced, and still liveA type outside types holding objects. Not a failure here, but it is why you might want types: 'all'.
+
+ +

Where the leak usually is

+ + +
+
+ + + + + + diff --git a/docs/extension.html b/docs/extension.html new file mode 100644 index 0000000..801957b --- /dev/null +++ b/docs/extension.html @@ -0,0 +1,446 @@ + + + + + +Browser extension — WebCodecs Census + + + + + + + +
+
+ WebCodecs Census docs + +
+ + + +
+
+
+ +
+
+

Browser extension

+

Look at a tab by hand

+

+ A Chrome extension for the case where you want to point at a running app rather than drive + one from a script. It is not published to the Chrome Web Store — build it and load it + unpacked. +

+ +

Nothing is instrumented until you say so

+ +

+ The extension ships no host permissions and no declared content scripts. + Enabling a site requests that one origin, registers a document_start content + script for it, and disabling unregisters the script and hands the permission back. +

+ +

+ A leak tool has no business patching WebCodecs on every page you visit. An earlier version + declared content scripts on <all_urls>, which patched the globals and + exposed the census API in the main world of every page — for a tool you only ever need on + one app. That was the wrong default however useful the tool is. +

+ +

+ Revoking the permission from Chrome's own settings also stops the content scripts: the + extension reconciles stored origins against granted permissions on install, on startup, and + whenever a permission is removed. +

+ +

Two modes

+ +

The exact one is not free, so you choose.

+ +
+ + + + + + + + + + +
Patch modeExact mode
Permissionsone origin, granted by youthat, plus debugger
Bannernone"debugging this browser"
Works with DevTools openyesno
Workers started before the page scriptmissedcaught
Workers blocked by worker-src CSPmissed (reported)caught
Codecs inside workerscaughtcaught
+
+ +

Patch mode

+ +

+ The census runs as a world: "MAIN" content script at + document_start, and rewrites new Worker(url) to load a small blob + that imports the census first and the real worker second. No debugger, no banner, and it + coexists with an open DevTools window. +

+ +

It is genuinely the weaker half, and it says so rather than pretending otherwise:

+ + + +

Exact mode

+ +

+ The same mechanism as the CDP driver, driven through + chrome.debugger so it works on a tab you are already looking at: auto-attach + pauses each worker, a beforeScriptExecution breakpoint gives the second, complete + pause, and the census goes in there. +

+ +

+ It costs a "debugging this browser" banner, and it cannot share a tab with an open DevTools + window — Chrome allows one debugger client per tab. +

+ +

+ Exact mode injects the plain census rather than the Worker-rewriting build: CDP reaches + workers on its own, so the rewrite would be redundant and would change + self.location for no gain. +

+ +

Build and load it

+ +
npm install
+npm run build:all      # then load extension/dist unpacked
+ +

+ In Chrome: chrome://extensions → enable Developer mode → Load + unpacked → pick extension/dist. Manifest V3, minimum Chrome 116. +

+ +

Using it

+ +
    +
  1. Open the app you want to measure and click the extension icon.
    2. +
  2. + Enable on host — Chrome asks for that one origin. Reload the + tab, so the census is in place before the app builds its decoders. +
    3. +
  3. Optionally Enable exact mode, then reload again so workers are caught at startup.
    4. +
  4. Refresh in the popup to read the census.
    5. +
+ +

The popup shows, per context: live counts by type, media elements and how many are stalled, + anything garbage collected without close(), the top allocation sites with two + stack frames each, and anything the install could not instrument. In patch mode it also + reports how many workers were wrapped and which were skipped, with the reason.

+ +
+ If nothing answers +

+ "No instrumented context answered" means either the site is not enabled, or the page + loaded before it was. Enable the site, then reload the tab. +

+
+ +

What it can see about the page

+ +

+ The census API is a page global — window.__webcodecsCensus — readable by any + script in the document, including third-party ones. Everything it exposes is already + derivable in-page, but treat it as visible rather than private, and do not enable it on a + page handling data you would not want an analytics script to see. Allocation stacks contain + your source URLs, function names and line numbers. +

+
+
+ + + + + + diff --git a/docs/index.html b/docs/index.html new file mode 100644 index 0000000..7f1b650 --- /dev/null +++ b/docs/index.html @@ -0,0 +1,608 @@ + + + + + +WebCodecs Census + + + + + + + +
+
+ WebCodecs Census docs + +
+ + + +
+
+
+ +
+
+

WebCodecs Census

+

Find the frame you forgot to close.

+

+ A leak detector for WebCodecs apps. It counts every VideoFrame, + AudioData and codec a page holds open — including the ones inside a + Web Worker — and hands back the line of code that allocated them. +

+ +
+ +

Quickstart →

+

Install, attach, read a leak report. Three ways in.

+ + +

API reference →

+

checkLeaks, expectNoLeaks, and what a report means.

+ + +

CI recipes →

+

Make a leak a test failure instead of a mystery.

+ + +

Honest limits →

+

What it cannot see, and what it says instead of guessing.

+ +
+ +

The problem

+ +

+ WebCodecs objects hold resources from a finite pool outside the JS heap. The garbage + collector never reclaims them; only close() does. Chrome's own guidance is + blunt about the consequence: forget frame.close() and you leak GPU memory + fast. +

+

+ Nothing in the platform tells you that you leaked one, how many, or + where from. The app just gets slower, then quietly stops decoding. DevTools' + Media panel lists media players; it does not attribute frame lifetimes. As far as we can + find, nothing else does either. +

+ +

What you get

+ +

One call, structured output, no screenshots:

+ +
import { attach, launchChrome } from '@motionvector/webcodecs-census-cdp';
+import { summarize } from '@motionvector/webcodecs-census';
+
+const chrome = await launchChrome({ executablePath: CHROME });
+const session = await attach({ browserURL: chrome.browserURL });
+await session.navigate('http://localhost:5173/');
+
+console.log(summarize(await session.census()));   // page + every worker
+ +
2 context(s): main, worker
+  main (3s): nothing live
+  worker (3s): VideoDecoder=1 VideoFrame=5
+
+5 VideoFrame still live (allowed 0).
+
+Held by:
+  5x VideoFrame (decoded, oldest 100ms) in worker
+      at VideoSample.toVideoFrame (pipeline.js:17261:14)
+      (frame emitted by this VideoDecoder)
+ +

+ The last three lines are the whole point. A count alone cannot be acted on; an allocation + site can. +

+ +

Why this is hard, and why it didn't exist

+ +

+ Decoders almost always live in a Web Worker. Page-level monkey-patching — how + Spector.js and + WebGPU Inspector both + work — cannot reach a worker the page created. That is the whole problem, and it has two + teeth: +

+ +

+ Either failure is silent, and a leak detector that silently sees nothing reports a clean + bill of health for an app that is losing every frame. That is worse than having no tool. +

+ +

The two-phase injection

+ +

+ Target.setAutoAttach with waitForDebuggerOnStart pauses a worker + before its first line. Inject there and you catch an allocation on line 1. But at that + moment a dedicated worker's global is only half built. Measured on Chrome 151: +

+ +
+ + + + + + + + + +
At the auto-attach pausePresentAbsent
VideoFrame, AudioData, ImageBitmap, EncodedVideoChunkyes
VideoDecoder, VideoEncoder, AudioDecoder, AudioEncoderabsent
setInterval, setTimeout, queueMicrotaskabsent
+
+ +

+ Patch there and you instrument the frame types but miss every codec — most + of what the tool is for. So the driver resumes into a second, later pause: a + beforeScriptExecution instrumentation breakpoint. That fires with the global + fully populated and still before the worker's own script runs. It is the only moment that + is both complete and early enough. +

+ +
+ Sequence diagram of the two-phase injection: the driver arms auto-attach, Chrome pauses the new worker before its first line where codecs and timers do not yet exist, the driver sets a beforeScriptExecution breakpoint and resumes into a second pause where the global is complete, and injects the census there. + Sequence diagram of the two-phase injection: the driver arms auto-attach, Chrome pauses the new worker before its first line where codecs and timers do not yet exist, the driver sets a beforeScriptExecution breakpoint and resumes into a second pause where the global is complete, and injects the census there. +
Two pauses per worker. The first is early but incomplete; the second is both.
+
+ +

+ Auto-attach is not recursive, so each attached target arms it again for its children. That + is what reaches nested workers. This behaviour is measured rather than specified, so + test/platform-assumptions.test.mjs asserts it directly and prints what it + found — a Chrome change is reported as a Chrome change rather than surfacing as a + mysterious failure elsewhere. +

+ +

Decoded frames never pass through a constructor

+ +

+ The frames that leak in production are not the ones you build with + new VideoFrame(). They are created by the platform and handed to the + output callback you gave new VideoDecoder({ output }). An + instrument that only traps the constructor counts a handful of hand-built frames and + misses the entire decode pipeline. +

+

+ The census wraps the output callback at construction time. Because there are + no application frames above a platform callback, a decoded frame is attributed to + the decoder that produced it — which is the line you can act on. + .clone() is tracked too: it returns an independent handle needing its own + close(), and it also bypasses the constructor. +

+ +

Why not just take a heap snapshot?

+ +

+ Chrome's DevTools MCP + server gained heap-snapshot tools for agents in Chrome 151, which is the natural thing + to reach for. It cannot answer this question, for three structural reasons rather than one + fixable one. Measured against this repository's own fixture, which leaks five + VideoFrames inside a worker: +

+ +
+ + + + + +
Heap snapshot of the page target4 VideoFrame nodes, 84 bytes total
webcodecs_censusVideoFrame: 5, attributed to the decoder that produced them
+
+ +
    +
  1. + Wrong heap. A WebCodecs object's resource lives outside the JS heap — + the entire reason close() exists. The snapshot measures JS wrappers, so it + understates a frame holding megabytes of GPU memory as tens of bytes. +
    2. +
  2. + Wrong scope. The leak is in a worker, and a page-target snapshot does + not cover worker isolates. +
    3. +
  3. + Wrong question. A frame collected by GC without + close() is the most definitive leak there is, and it is gone from the heap + by the time you could snapshot it. Only a FinalizationRegistry sees it, + which is what this does. +
    4. +
+ +

+ The two compose rather than compete: several CDP clients can attach to one page at the + same time, so an agent can run Chrome's DevTools MCP for JS-heap and performance work and + this one for media object lifetimes. +

+ +

What it tracks

+ +

+ VideoDecoder, VideoEncoder, AudioDecoder, + AudioEncoder, VideoFrame, AudioData, + ImageBitmap — plus <video> and <audio> + elements. Each live object records how it entered the context, because provenance decides + whether a leak is yours: +

+ +
+ + + + + + + + +
OriginMeaning
constructednew VideoFrame(...) here
decodedproduced by a codec, attributed to that codec's construction site
cloned.clone() — an independent handle needing its own close()
receivedarrived over postMessage; this context owns it now
+
+ +

+ Departures are accounted for just as carefully. Transferring a VideoFrame + detaches the sender's handle without calling close(), and the receiver + gets it by structured clone rather than a constructor — counted naively that is a false leak + in one context and an invisible object in the other. A FinalizationRegistry + catches the unambiguous case: an object collected by GC that was never closed. And since + v0.3.0, a codec the platform closed after an error is recognised as gone rather than filed + as leaked, because the spec closes it before your error callback runs. +

+ +

Who this is for

+ +

+ Anything decoding or encoding in the browser: timeline editors, recorders, transcoders, + players that seek by decoding. + Clipchamp + presented its WebCodecs pipeline at a W3C workshop, and + Remotion has folded its media parser + into mediabunny and now recommends it. The more of your pipeline lives in a library, + the more of it an app-only instrument cannot see. +

+ +
+ Works through libraries +

+ If you use a toolkit like mediabunny, + you never write new VideoDecoder — the library does. The census patches the + globals, and mediabunny references them at call time rather than capturing them at module + scope, so everything it builds internally is counted. + test/mediabunny.test.mjs builds a real MP4 with mediabunny, decodes it back + through its own sinks, and asserts that the leak lands on the exact method whose + double-ownership contract was broken. +

+
+ +

Three packages, one version

+ +
+ + + + + + + + + + + + + + + + +
PackageWhat it is
@motionvector/webcodecs-censusThe instrumentation core and the assertion API. No dependencies. Reference
@motionvector/webcodecs-census-cdpInjects the census into a running Chrome, workers included. Reference
@motionvector/webcodecs-census-mcpAn MCP server, so an agent can do all of the above. Reference
+
+ +

+ They share one version, because -cdp and -mcp depend on an + exact version of the core. A Chrome extension covers + the case where you want to look at a tab by hand. +

+ +

Prior art

+ + +
+
+ + + + + + diff --git a/docs/limits.html b/docs/limits.html new file mode 100644 index 0000000..23140a8 --- /dev/null +++ b/docs/limits.html @@ -0,0 +1,499 @@ + + + + + +Honest limits — WebCodecs Census + + + + + + + +
+
+ WebCodecs Census docs + +
+ + + +
+
+
+ +
+
+

Honest limits

+

What it cannot see

+

+ A leak detector that silently sees nothing reports a clean bill of health for an app that is + losing every frame. That is worse than having no tool. So here is the blind-spot list, and + the machinery that keeps a gap from passing as a pass. +

+ +

How a gap is reported

+ +
+ + + + + + + + + + + + + + + + +
SignalMeans
problems[] + A patch step that failed in this context. Every step of + installCensus is wrapped separately and records its failure here + instead of throwing. Non-empty means the counts beside it are a floor, not a + total. +
closedUnseen + Something was closed here that the census never saw arrive. Usually a receive path + the message scanner did not reach. It is a count of objects whose lifetime was only + half observed. +
skipped workers (patch mode) + Every worker the extension's patch mode could not wrap, with the reason, in the + popup and in the census payload. +
+
+ +

+ If you add a code path that can silently observe less than it appears to, add the counter or + the problems[] entry that makes it visible. That is the project's one + non-negotiable rule. +

+ +

Blind spots

+ +

Anything allocated before it installs

+ +

+ The census counts what it sees enter a context. Objects that already existed are invisible — + not counted live, and a later close() on one lands in + closedUnseen. This is the entire reason the CDP driver goes to the trouble of + two pauses per worker. +

+ +

Encoded chunks

+ +

+ EncodedVideoChunk and EncodedAudioChunk have no + close() and hold no external resource, so they are not tracked as leakable. + Encoders emit them, which is why an encoder's output callback produces nothing + the census counts. +

+ +

Frames from MediaStreamTrackProcessor

+ +

+ They reach you through a ReadableStream rather than a constructor or a codec + output callback, so they are not attributed to an allocation site. Closing one + shows up as closedUnseen rather than as a matched lifetime. +

+ +

Objects buried deep in a message

+ +

+ The receive-side scanner walks 3 levels and 64 entries per + level, and descends only into arrays and plain object carriers — an object whose prototype is + a class is not walked into. This runs on every message and has to stay cheap. A frame buried + deeper, or held on a class instance, arrives uncounted and later surfaces as + closedUnseen. +

+ +

Patch mode changes self.location

+ +

+ A worker wrapped by the extension's patch mode sees the loader blob URL as + self.location. Workers using import.meta.url are unaffected; + workers building paths from self.location are not. Patch mode also cannot cover + a worker that started before it installed, and cannot create a blob worker at all on a page + whose CSP omits blob: — which it reports rather than hides. +

+ +

Exact mode cannot share a tab with DevTools

+ +

Chrome allows one debugger client per tab. Patch mode exists for when that trade is not worth it.

+ +

One context at a time, in-process

+ +

+ localCensus() covers only the context it runs in. Getting a worker's census + means running it inside that worker and carrying the result back. The CDP driver and the + extension do this by querying each target on its own session. +

+ +

collectedUnclosed waits for the collector

+ +

+ It is reported by a FinalizationRegistry, which fires when the GC actually + collects the object. That is not on your schedule. A leak that has not been collected yet + shows up as live, not as collectedUnclosed — the two are the same + bug at different stages. +

+ +

An age filter is exact only up to the census cap

+ +

+ minAgeMs decides the verdict as of v0.3.1, using the ages the census carries. + Those ages are capped at 256 per type, kept from the oldest end. Below saturation the count + is exact; at saturation — every kept age clears the threshold — the report states a lower + bound and carries the exact total beside it, because the objects past the cap are of unknown + age. It never under-reports, and it never claims more than it knows. +

+ +

+ Before v0.3.1 the option filtered report.sites only, so it changed the printed + attribution without changing pass or fail. If you are pinned to an older release, use + allow instead. +

+ +

Chrome, for the driver and the extension

+ +

+ The CDP driver and the extension are Chrome and Chromium only — they depend on the DevTools + Protocol and on Chrome-specific worker pause behaviour. The core is plain JavaScript with no + dependencies and runs anywhere WebCodecs does. +

+ +

Undocumented behaviour, pinned deliberately

+ +

+ Injection depends on Chrome behaviour that is measured rather than specified: that workers + pause before their first line, that the beforeScriptExecution breakpoint + exists, and that codecs are absent at the earlier pause. + test/platform-assumptions.test.mjs asserts each of those and prints what it + measured, so a browser change is reported as a browser change. CI pins one Chrome version + for its blocking job and runs a separate scheduled job against stable — drift is news, not a + blocked pull request. +

+ +

What it costs

+ +

+ Measured, not estimated: +5.6 µs per tracked allocation and + ~284 bytes per live tracked object, the latter bounded by the size of the + leak itself. At 60 fps that is 0.03% of a second. It only matters above roughly 100k + allocations per second. +

+ +

+ Suitable for development and CI. Not recommended enabled by default in production builds — + not for speed, but because it patches global constructors and retains a stack per live + object. +

+ +

What it deliberately does not do

+ + + +

Things that look alarming and are the point

+ +

+ The census patches global constructors, keeps allocation stacks in memory, exposes + window.__webcodecsCensus to anything in the page, and the MCP server runs + arbitrary JavaScript in the page under test. CDP ports are unauthenticated by design. None + of that is incidental — + SECURITY.md + says which of it is deliberate, which is not, and how to report a problem. +

+
+
+ + + + + + diff --git a/docs/mcp.html b/docs/mcp.html new file mode 100644 index 0000000..620ca4c --- /dev/null +++ b/docs/mcp.html @@ -0,0 +1,480 @@ + + + + + +MCP server — WebCodecs Census + + + + + + + +
+
+ WebCodecs Census docs + +
+ + + +
+
+
+ +
+
+

MCP server

+

@motionvector/webcodecs-census-mcp

+

+ Let an agent find a leaked VideoFrame — with the allocation stack — without a + browser UI in the loop. +

+ +

+ Existing media tooling is built for human eyes: panels, flamegraphs, screenshots. An agent + cannot read those efficiently. This exposes the same information as small, structured text. +

+ +

Register it

+ +
{
+  "mcpServers": {
+    "webcodecs-census": {
+      "command": "npx",
+      "args": ["-y", "@motionvector/webcodecs-census-mcp"]
+    }
+  }
+}
+ +

+ It speaks stdio and holds one session at a time. SIGINT and + SIGTERM detach and close any browser it launched. +

+ +

Tools

+ +
+ + + + + + + + + + +
ToolWhat it answers
webcodecs_attachInstrument a page and every one of its Web Workers. Call first.
webcodecs_censusWhat is open right now, as a digest.
webcodecs_leak_sitesWhich line is leaking — grouped by allocation site, worst first.
webcodecs_timelineLive counts, throughput, queue depth and media readiness over time.
webcodecs_evaluateDrive the app so the census has activity to observe.
webcodecs_detachStop, and close any browser this launched.
+
+ +

webcodecs_attach

+ +
+ + + + + + + + + +
ParameterTypeDefaultWhat it does
urlstring—Page to open and instrument. The server navigates, then settles briefly before returning.
browserURLstring—DevTools endpoint of a Chrome already started with --remote-debugging-port.
executablePathstring—Chrome binary to launch instead, with a throwaway profile.
headlessbooleantrueOnly meaningful with executablePath.
sampleIntervalMsnumber250Timeline sampling interval. The server keeps 400 samples per context.
+
+ +

+ One of browserURL or executablePath is required. Calling it while + already attached returns a note rather than replacing the session — detach first. +

+ +

webcodecs_census

+ +

+ Returns summarize() for every instrumented context, plus anything that could + not be instrumented, under a Not instrumented: heading. Small by design — use + webcodecs_leak_sites for stacks. +

+ +

Takes waitMs (default 0) to settle before sampling.

+ +

webcodecs_leak_sites

+ +
+ + + + + + + +
ParameterDefaultWhat it does
typeevery tracked typeRestrict to one type, e.g. VideoFrame.
limit10Maximum sites returned.
minAgeMs0Ignore sites whose oldest object is younger than this.
+
+ +
+ Attribution is not a verdict +

+ This tool covers every tracked type unless you name one. It deliberately + does not inherit the assertion API's default of frame-like types only: an agent asking + "which line is leaking" about a codec leak would otherwise get nothing back. +

+
+ +

webcodecs_timeline

+ +

+ Takes context (restrict to one context name) and lastN (default + 40). +

+ +
webcodecs_timeline { context: "worker", lastN: 20 }
+
+### worker (20 samples)
+  t(ms)  live      dec/out  queued  media(stalled)
+  12000  VF=59     0/0      0       4(0)
+  58000  VF=58     0/0      0       4(1)
+ 162000  VF=58     0/0      0       4(1)
+ +

+ VF is VideoFrame; other types abbreviate to their initial. + dec/out is decode calls over output callbacks in that interval, and + queued is summed codec queue depth. +

+ +

+ Decoder idle, queue empty, count frozen: wedged, not buffering. A single snapshot cannot + tell those apart, and the difference is between chasing a resource limit that does not + exist and finding the actual bug. +

+ +

webcodecs_evaluate

+ +

+ Runs an expression in the page — click something, start playback — so the census has + activity to observe. Takes expression, which is required. Returns the value as + JSON. +

+ +

webcodecs_detach

+ +

Stops instrumenting and closes any browser the server launched. No parameters.

+ +

A session

+ +
webcodecs_attach { executablePath: "/path/to/chrome", url: "http://localhost:5173/" }
+→ Attached.
+  Instrumented 3 context(s):
+    page   http://localhost:5173/
+    worker http://localhost:5173/decode-worker.js
+    worker blob:http://localhost:5173/…
+  Workers are instrumented before their first line, so allocations at worker
+  startup are counted.
+
+webcodecs_census
+→ worker (20s): VideoDecoder=1 VideoFrame=58
+  1 VideoFrame garbage collected without close() — definitively leaked.
+
+webcodecs_leak_sites { type: "VideoFrame" }
+→ 58x VideoFrame — decoded, oldest 124609ms, in worker
+      at PackagerWorker.setupDecoder (worker.js:1756:21)
+      (frame emitted by this VideoDecoder)
+ +

Running alongside chrome-devtools-mcp

+ +

+ Several CDP clients can attach to one page at the same time, so this runs happily beside + chrome-devtools-mcp. + They cover different ground: heap snapshots measure the JS heap, and a WebCodecs resource + lives outside it — a page-target snapshot reports tens of bytes for frames holding + megabytes of GPU memory, does not cover worker isolates, and cannot see a frame GC already + collected without close(). +

+ +
+ Trust boundary +

+ webcodecs_evaluate runs arbitrary JavaScript in the page under test — that + is how an agent drives the app it is measuring. Anything driving this server can run code + in any page it attaches to. Point it at applications you control, and never expose a + debugging port beyond localhost. +

+
+
+
+ + + + + + diff --git a/docs/quickstart.html b/docs/quickstart.html new file mode 100644 index 0000000..1eda3c7 --- /dev/null +++ b/docs/quickstart.html @@ -0,0 +1,478 @@ + + + + + +Quickstart — WebCodecs Census + + + + + + + +
+
+ WebCodecs Census docs + +
+ + + +
+
+
+ +
+
+

Quickstart

+

Get a leak report

+

+ Three ways in, depending on whether you can edit the code being measured, want to drive a + browser from a script, or want an agent to do it. +

+ +
+ The one thing that matters +

+ The census only counts what it sees enter a context. Anything allocated before + it installs is invisible to it. Install as early as you can — which is exactly why the + CDP driver exists: it gets in before a worker's first line, which nothing running inside + the page can do. +

+
+ +

1. From your own code

+ +

Reach for this when you can edit the app or the test harness.

+ +
npm install --save-dev @motionvector/webcodecs-census
+ +

Install it at the top of every context that touches media — the main thread and each worker:

+ +
import { installCensus, localCensus } from '@motionvector/webcodecs-census';
+
+installCensus({ context: 'decoder-worker' });
+ +

Then ask what is still open:

+ +
const census = localCensus();
+// {
+//   live:              { VideoFrame: 58, VideoDecoder: 1 },
+//   entered:           { 'VideoFrame:decoded': 238, 'VideoDecoder:constructed': 1 },
+//   left:              { 'VideoFrame:closed': 179 },
+//   leakSites:         [ { count: 58, type: 'VideoFrame', origin: 'decoded', stack, oldestAgeMs } ],
+//   collectedUnclosed: { VideoFrame: 1 },
+//   mediaElements:     { total: 4, stalled: 1, byReadyState: { 0: 1, 4: 3 } },
+//   timeline:          [ … ],
+//   problems:          [],
+// }
+ +

+ localCensus() covers only the context it runs in. A worker's + census has to be collected in that worker and carried back to whoever is asking — the CDP + driver does this for you by querying each target on its own session. +

+ +

2. From outside the app

+ +

+ No change to the code being measured, and workers are instrumented before their first line. +

+ +
npm install --save-dev @motionvector/webcodecs-census-cdp
+ +
import { attach, launchChrome } from '@motionvector/webcodecs-census-cdp';
+import { summarize } from '@motionvector/webcodecs-census';
+
+const chrome = await launchChrome({ executablePath: CHROME });
+const session = await attach({ browserURL: chrome.browserURL });
+
+await session.navigate('http://localhost:5173/');
+await session.evaluate('document.querySelector("#play").click()');
+// …let it run…
+
+console.log(summarize(await session.census()));
+
+session.detach();
+await chrome.kill();
+ +

+ To attach to a browser you already started with --remote-debugging-port, pass + its endpoint instead: +

+ +
const session = await attach({ browserURL: 'http://127.0.0.1:9222' });
+ +

Full options on the CDP driver page.

+ +

3. From an agent

+ +

Register the MCP server with whatever runs your agent:

+ +
{
+  "mcpServers": {
+    "webcodecs-census": {
+      "command": "npx",
+      "args": ["-y", "@motionvector/webcodecs-census-mcp"]
+    }
+  }
+}
+ +
webcodecs_attach { executablePath: "/path/to/chrome", url: "http://localhost:5173/" }
+→ Instrumented 3 context(s): page, worker, worker
+
+webcodecs_census
+→ worker (20s): VideoDecoder=1 VideoFrame=58
+  1 VideoFrame garbage collected without close() — definitively leaked.
+
+webcodecs_leak_sites { type: "VideoFrame" }
+→ 58x VideoFrame — decoded, oldest 124609ms, in worker
+      at PackagerWorker.setupDecoder (worker.js:1756:21)
+      (frame emitted by this VideoDecoder)
+ +

Tool-by-tool detail on the MCP server page.

+ +

Reading the report

+ +

These mean different things, and the difference is worth internalising.

+ +
+ + + + + + + + + + + + + + + + +
SignalWhat it means
live + Still open right now. Could be a leak, could be a pipeline mid-flight. Judge it + against how long they have been live and how many you expect. +
collectedUnclosed + Garbage collected while still open. The resource was held for the object's whole + lifetime and nothing will ever release it. Not a heuristic — a leak. +
overCloses (v0.3.0) + A close() that threw, because something closed an already-closed codec. + A lifecycle bug, not a leak — reported, and only fatal if you ask for it. +
+
+ +

+ That is why collectedUnclosed fails a check for any tracked type, whatever you + passed for types — and why an over-close does not. A codec the platform closed + after an error is not counted as either. See the API reference. +

+ +

Making it a test

+ +
import { expectNoLeakedFrames } from '@motionvector/webcodecs-census';
+
+test('the editor releases every frame it decodes', async () => {
+  await playThroughTimeline();
+  expectNoLeakedFrames(await session.census());
+});
+ +

+ It throws with the allocation sites attached. CI recipes covers + running this against a real Chrome on a runner. +

+ +

Next

+ +
+ +

API reference →

+

Every function, every option, and what a verdict is computed from.

+ + +

Honest limits →

+

What the census cannot see, and how it tells you so.

+ +
+
+
+ + + + + + diff --git a/packages/cdp/README.md b/packages/cdp/README.md index b0e8b3f..028b210 100644 --- a/packages/cdp/README.md +++ b/packages/cdp/README.md @@ -110,7 +110,10 @@ localhost. ## Documentation -Full documentation, the extension, and the MCP server: -**[github.com/motionvector-dev/webcodecs-census](https://github.com/motionvector-dev/webcodecs-census)** +Full reference, the extension, and the MCP server: +**[motionvector-dev.github.io/webcodecs-census](https://motionvector-dev.github.io/webcodecs-census/cdp.html)** + +Source and issues: +[github.com/motionvector-dev/webcodecs-census](https://github.com/motionvector-dev/webcodecs-census) MIT diff --git a/packages/core/README.md b/packages/core/README.md index 221cf30..389249b 100644 --- a/packages/core/README.md +++ b/packages/core/README.md @@ -213,7 +213,10 @@ release is traceable to the commit and workflow that built it. ## Documentation -Full documentation, the two-phase worker injection, and the honest limits: -**[github.com/motionvector-dev/webcodecs-census](https://github.com/motionvector-dev/webcodecs-census)** +Full reference, the two-phase worker injection, and the honest limits: +**[motionvector-dev.github.io/webcodecs-census](https://motionvector-dev.github.io/webcodecs-census/)** + +Source and issues: +[github.com/motionvector-dev/webcodecs-census](https://github.com/motionvector-dev/webcodecs-census) MIT diff --git a/packages/mcp/README.md b/packages/mcp/README.md index 12bc024..1ef78fc 100644 --- a/packages/mcp/README.md +++ b/packages/mcp/README.md @@ -100,6 +100,10 @@ debugging port beyond localhost. ## Documentation -**[github.com/motionvector-dev/webcodecs-census](https://github.com/motionvector-dev/webcodecs-census)** +Full reference, tool by tool: +**[motionvector-dev.github.io/webcodecs-census](https://motionvector-dev.github.io/webcodecs-census/mcp.html)** + +Source and issues: +[github.com/motionvector-dev/webcodecs-census](https://github.com/motionvector-dev/webcodecs-census) MIT