Found while writing the 2.0.0 CHANGELOG (#268), verified independently.
The problem
SnapDiff::Reporting.register appears exactly once in the gem, at lib/snap_diff/reporters/html.rb:140. So the honest summary line comes bundled with the HTML report. A user who follows our documented Rails setup gets neither:
$ ruby -e 'require "snap_diff/integrations/minitest"; puts SnapDiff::Reporting.reporters.size'
0
lib/snap_diff/integrations/minitest.rb requires dsl, screenshot_assertion and reporting — never reporters/html. So out of the box there is no reporter, and therefore no summary.
Why this matters more than a missing feature
The summary line exists specifically to catch the failure modes no per-assertion rule can see. Of the three routes to a green suite that compared nothing, two — rake test running zero system tests, and an inherited GIT_DIR redirecting every baseline lookup — never reach an assertion at all. 0 verified is the only signal for either.
Shipping that signal behind an opt-in require means the default configuration is the one that cannot detect the problem the signal was built for.
A 1.15.1 user upgrading also sees the headline improvement of #261 as simply absent.
The distinction to make
The counting is core honesty; the HTML file is a feature. They are currently one object.
- The summary line is stdout, costs nothing, and should always print.
- Writing
snap_diff_report.html creates a file on disk — reasonable to keep opt-in, since turning it on for everyone changes what a test run leaves behind.
Proposed: SnapDiff::Reporting owns the counting and prints the summary unconditionally; Reporters::HTML keeps the report file and stays opt-in. Today the second one owns both.
Checklist
Related, same root cause
docs/reporters.md documents record/finalize/summary as the custom-reporter contract. If counting moves to Reporting, that contract gets simpler — a custom reporter stops being responsible for telling the truth about what ran.
Found while writing the 2.0.0 CHANGELOG (#268), verified independently.
The problem
SnapDiff::Reporting.registerappears exactly once in the gem, atlib/snap_diff/reporters/html.rb:140. So the honest summary line comes bundled with the HTML report. A user who follows our documented Rails setup gets neither:lib/snap_diff/integrations/minitest.rbrequiresdsl,screenshot_assertionandreporting— neverreporters/html. So out of the box there is no reporter, and therefore no summary.Why this matters more than a missing feature
The summary line exists specifically to catch the failure modes no per-assertion rule can see. Of the three routes to a green suite that compared nothing, two —
rake testrunning zero system tests, and an inheritedGIT_DIRredirecting every baseline lookup — never reach an assertion at all.0 verifiedis the only signal for either.Shipping that signal behind an opt-in
requiremeans the default configuration is the one that cannot detect the problem the signal was built for.A 1.15.1 user upgrading also sees the headline improvement of #261 as simply absent.
The distinction to make
The counting is core honesty; the HTML file is a feature. They are currently one object.
snap_diff_report.htmlcreates a file on disk — reasonable to keep opt-in, since turning it on for everyone changes what a test run leaves behind.Proposed:
SnapDiff::Reportingowns the counting and prints the summary unconditionally;Reporters::HTMLkeeps the report file and stays opt-in. Today the second one owns both.Checklist
Reporters::HTMLintoReporting0 verifiedstill shouts (theNOTHING WAS VERIFIEDcase, added in feat: the summary line counts what was verified, not just what was compared #261)Reportingalready owns the fragment merge, so this should simplify rather than complicate itRelated, same root cause
docs/reporters.mddocumentsrecord/finalize/summaryas the custom-reporter contract. If counting moves toReporting, that contract gets simpler — a custom reporter stops being responsible for telling the truth about what ran.