Split out of #284 so it does not sit in the release path.
BrowserScreenshotTest#test_compare_crops_only_when_other_part_is_not_working fails
intermittently on one matrix cell, Test Drivers (selenium_headless, chunky_png):
the change spans 4378 of 16800 px (26.06% of the 200x84 image)
changed region: [0,19,199,41] (left,top,right,bottom edges)
max color distance: 50.9
judged against: tolerance 0.13, color_distance_limit 40
What is established
- Base rate: 4 success / 1 failure over the last 12
Test runs on that cell. Every
other Drivers cell is 100%.
- Not environment drift. The failing job (
97473161378) and a passing job at the same
commit (97450518382) ran on the identical runner image 20260816.277.1, same
runner version, same Ruby.
- Not antialiasing. The changed region is 200x22 = 4400 px against 4378 px changed —
the entire first input row and nothing else — at max colour distance 50.9. That is a
materially different render of #first-field, not sub-pixel drift.
- The fixture (
test/fixtures/app/index-without-img.html) loads no remote resources;
the font stack is local with fallbacks, so it is not the usual web-font race.
What is not established
The actual cause. A plausible shape is leftover focus/selection state on #first-field
from the test that does fill_in "First Field:", but that is unverified — and it
cannot be verified from CI artifacts until #284 is fixed, because the failure's own
evidence is overwritten before upload.
Blocked on #284
Fix the shared screenshot name first, then let this flake reproduce once with real
artifacts. Debugging it before that means guessing.
Also worth deciding
fail-fast turns this single flaky cell into three red cells (two siblings report
cancelled), so a run reports far more damage than occurred. Same mechanism hid the JRuby
lane for 15 runs (#283).
Scheduling
Not a 2.0.0 blocker: it predates the release, and chunky_png is deprecated in 2.0 and
removed in 2.1 — so this cell disappears on its own in 2.1. Filed against 2.1.0 so the
deletion is a deliberate outcome rather than an accidental one, and so nobody mistakes the
removal for a fix.
Split out of #284 so it does not sit in the release path.
BrowserScreenshotTest#test_compare_crops_only_when_other_part_is_not_workingfailsintermittently on one matrix cell,
Test Drivers (selenium_headless, chunky_png):What is established
Testruns on that cell. Everyother Drivers cell is 100%.
97473161378) and a passing job at the samecommit (
97450518382) ran on the identical runner image20260816.277.1, samerunner version, same Ruby.
the entire first input row and nothing else — at max colour distance 50.9. That is a
materially different render of
#first-field, not sub-pixel drift.test/fixtures/app/index-without-img.html) loads no remote resources;the font stack is local with fallbacks, so it is not the usual web-font race.
What is not established
The actual cause. A plausible shape is leftover focus/selection state on
#first-fieldfrom the test that does
fill_in "First Field:", but that is unverified — and itcannot be verified from CI artifacts until #284 is fixed, because the failure's own
evidence is overwritten before upload.
Blocked on #284
Fix the shared screenshot name first, then let this flake reproduce once with real
artifacts. Debugging it before that means guessing.
Also worth deciding
fail-fastturns this single flaky cell into three red cells (two siblings reportcancelled), so a run reports far more damage than occurred. Same mechanism hid the JRubylane for 15 runs (#283).
Scheduling
Not a 2.0.0 blocker: it predates the release, and
chunky_pngis deprecated in 2.0 andremoved in 2.1 — so this cell disappears on its own in 2.1. Filed against 2.1.0 so the
deletion is a deliberate outcome rather than an accidental one, and so nobody mistakes the
removal for a fix.