Skip to content

The gem does not wait for webfonts, so users mask the font-swap race with tolerances and retries #273

Description

@pftg

Third finding from a real project's workaround layer (jetthoughts.github.io). Unscheduled — recorded now so a kickoff has the evidence.

What users have to build themselves

# Fonts settled before capture: a webfont swapping in mid-capture reflows
# text bimodally (the "font-swap race" this suite otherwise masks with
# skip_areas, 0.03 tolerances, and stability retries). fonts.ready
# resolves instantly once fonts are cached, so warm sessions pay ~0.
page.evaluate_async_script("var done = arguments[0]; document.fonts.ready.then(function() { done(true) })")

One line, standard browser API, resolves instantly when fonts are cached. Without it, a webfont swapping in mid-capture reflows text bimodally — the screenshot is one of two valid layouts depending on timing.

The important part is the parenthetical: the alternative they were living with was skip_areas, 0.03 tolerances, and stability retries. Every one of those is expensive, and every one degrades the test's ability to catch real regressions — a 0.03 tolerance on a 1920x1080 frame lets ~62,000 pixels change before failing.

So a race we could eliminate is currently paid for in three separate forms of blindness.

Related, same class: lazy-loaded images

def preload_all_images
  scroll_to :bottom
  assert_text "JetThoughts. All Rights Reserved", exact: false
  scroll_to :top
end

A hand-rolled scroll-to-bottom-and-back to force lazy images to load before capture. Also standard, also reinvented per project.

Measured cross-platform noise, for context

From the same file, after measurement:

Measured run-to-run noise is ~1e-6 (three runs: 0.013838252 twice, then 0.013837770 — about 2 px of 2,073,600) … That 1e-6 is macOS local; two CI recordings of the same commit left 135 of 147 baselines byte-identical but moved 11 font/SVG-heavy pages by 0.0003–0.002, every one of which pins 0.03.

Local noise is ~2 px. On CI, font- and SVG-heavy pages move 300–2000× more. The pages that move are the font-heavy ones — which is what a font-swap race predicts.

Questions a kickoff should answer, not assume

  • Should document.fonts.ready be awaited before every capture by default? It is instant when fonts are cached, but it is still a synchronous browser round-trip and not every driver supports evaluate_async_script equally.
  • Is this better as a default, an opt-in capture option, or a documented recipe? The evidence that users reinvent it argues against "documented recipe" — the whole point is that they had to discover it.
  • Do the other stabilisation sources deserve the same treatment (lazy images, CSS animations, scroll-behavior, caret blink)? Two independent users were previously observed hand-rolling stabilising CSS injection.
  • Would this make stability_time_limit tunable down, which is the dominant per-screenshot cost (Instability is undiagnosable, so people add sleeps instead — and the wait is the dominant cost #271)?

Not proposing an implementation

Per the standing rule, this is a maintainer-adjacent idea until a kickoff decides its shape. What is not in doubt is the evidence: a real project reinvented it, measured the noise, and is paying for the race with tolerances that blind the tests.

Related: #271 (instability is undiagnosable, so people add sleeps), #272 (skip_area selectors block 5s each).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementneeds-evidenceMaintainer idea; unproven. Needs demand or measurement before scheduling.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions