diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 2958c4ad..1b4195fb 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -53,7 +53,7 @@ If applicable, add the `.diff.png` or `.heatmap.diff.png` files to help explain If applicable, run with `DEBUG=1` and paste the output: ``` -DEBUG=1 bundle exec rake test +bin/rails test:system # or whatever task runs your Capybara tests ``` ## Additional context diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index da99c0dc..0c07cc4f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -90,8 +90,8 @@ jobs: body: | ## What's Changed - See [CHANGELOG.md](https://github.com/${{ github.repository }}/blob/main/CHANGELOG.md#v${{ github.event.inputs.version }}) for full details. + See [CHANGELOG.md](https://github.com/${{ github.repository }}/blob/v${{ github.event.inputs.version }}/CHANGELOG.md) for full details. - **Upgrade Guide:** [docs/UPGRADING.md](https://github.com/${{ github.repository }}/blob/main/docs/UPGRADING.md) + **Upgrade Guide:** [docs/UPGRADING.md](https://github.com/${{ github.repository }}/blob/v${{ github.event.inputs.version }}/docs/UPGRADING.md) env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.gitignore b/.gitignore index d2a6596e..ca07113a 100644 --- a/.gitignore +++ b/.gitignore @@ -10,6 +10,7 @@ /gemfiles/*.lock /gems.locked /pkg/ +*.gem /spec/reports/ /tmp/ /vendor/sigs/ diff --git a/CHANGELOG.md b/CHANGELOG.md index ad54e1dc..d1024fb9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,186 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [v2.0.0] - unreleased + +**The transitional release.** Everything you run today keeps working. 2.0 makes +`SnapDiff` the canonical namespace, keeps the entire v1 API alongside it as +same-object aliases, and warns — once per process — about what **2.1** removes. +There is no 3.0; 2.1 is the cleanup. + +The sections below the divider are the prerelease notes (alpha1 → beta3) and are +kept as history. This entry is the one to read if you are coming from **1.15.1**. + +### Upgrading from 1.15.1: change the version, run your suite + +```ruby +gem "capybara-screenshot-diff", "2.0.0.beta3" # current 2.0 prerelease; 2.0.0 final is not out yet +``` + +Pin the exact prerelease until 2.0.0 ships: Bundler never resolves a prerelease from a plain +requirement, so `"~> 2.0"` fails with `Could not find gem 'capybara-screenshot-diff (~> 2.0)'`. +From 2.0.0 on, `"~> 2.0"` is the pin. + +That is the whole migration. `screenshot`, `assert_matches_screenshot` and +`capture_screenshot` are unchanged. Your `Capybara::Screenshot::Diff.configure` +block, every `Capybara::Screenshot.*` / `Capybara::Screenshot::Diff.*` setting, and +every legacy constant still resolve — to the *same objects* the new names resolve +to. Baselines are unchanged: capture, encoding, file naming and the `png` default are +the same code as 1.15.1, moved — upgrading does not re-encode or invalidate a baseline you +already committed, and a matching screenshot stays byte-identical. (Unchanged from 1.x: a +screenshot that *differs* is written to its baseline path, so a failing run leaves +`doc/screenshots/` dirty. That is how you accept a change — review the diff and commit.) + +Rolling back is a Gemfile edit: pin `"~> 1.15"` and `bundle update`. + +### What you will see in your test output + +One migration notice, the first time the process goes through a v1 door that *can* be +hooked — a legacy config accessor, `include Capybara::Screenshot[::Diff]`, +`Capybara::Screenshot::Diff.default_options`, or a lazily shimmed legacy constant: + +```text +[snap_diff deprecation] This process uses the v1 `Capybara::Screenshot*` / +`CapybaraScreenshotDiff*` API. It still works in 2.0 and is REMOVED in 2.1 -- +see docs/UPGRADING.md for the SnapDiff replacements. Silence with +`SnapDiff.silence_deprecations = true` or SNAP_DIFF_SILENCE_DEPRECATIONS=1. +(shown once per process) +``` + +Plus one line per *lazily shimmed* legacy constant you reference, naming your call +site: + +```text +[snap_diff deprecation] `Capybara::Screenshot::Diff::ImageCompare` is deprecated +(constant); use `SnapDiff::Comparison` instead. (called from test/test_helper.rb:12) +``` + +> **Silence is not evidence that you are migrated — on any 2.0 build.** In `2.0.0.beta3` +> the deprecation channel was incomplete: a v1-only suite got **no** warnings at all, and +> the driver-half removal warnings did not exist yet. 2.0.0 fixes the config-accessor, +> `include`, `default_options` and `const_missing` doors, but **a suite whose only contact +> with the v1 API is `require "capybara_screenshot_diff/minitest"` + +> `include CapybaraScreenshotDiff::Minitest::Assertions` still prints nothing** — those +> names are eager aliases, so there is no `const_missing` to hook. That setup is removed in +> 2.1 all the same. Do not use warning output as a migration checklist; use +> [docs/UPGRADING.md](docs/UPGRADING.md#deprecation-warnings), which lists what warns and +> what cannot. + +Requiring the gem, the DSL, settings accessors and the eagerly-defined constants +(the error classes, `::VERSION`, `Os`, `Region`, `Reporters::Default`, +`LOADED_DRIVERS`, `AVAILABLE_DRIVERS`) are **silent by design** — +[docs/UPGRADING.md](docs/UPGRADING.md#deprecation-warnings) lists exactly which +names warn and which do not. Silence everything with +`SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`. + +### The five things that can actually break + +Everything else is source-compatible. These are not: + +1. **Error class names printed in output are now `SnapDiff::…`.** The class objects + are identical, so `rescue CapybaraScreenshotDiff::ExpectationNotMet` still + catches them — but a CI job that greps the *old* class name out of test output + needs updating. +2. **`defined?` / `const_defined?` on lazily shimmed legacy names returns + `false`/`nil`.** They resolve through `const_missing`, which those checks never + trigger. Move feature detection to the `SnapDiff::` name. Names in the + silent-by-design list are real constants and are unaffected. +3. **Reopening `module Capybara::Screenshot::Diff::Drivers`** (the historical + custom-driver monkey-patch) defines a fresh, empty module that shadows the shim. + Define custom drivers under `SnapDiff::Drivers` instead — and `BaseDriver` is a + mixin now: `class MyDriver < BaseDriver` becomes `include SnapDiff::Driver`. +4. **Stubbing `Capybara::Screenshot::Diff::AVAILABLE_DRIVERS` no longer works.** It + is an eager alias; the gem reads `SnapDiff::Drivers::AVAILABLE_DRIVERS`. Stubbing + the legacy name rebinds the alias only, so the test passes for the wrong reason. +5. **`SnapDiff::Config::MAPPING` is gone**, split into `SnapDiff::Config::SETTINGS` + (setting names) and the `@api private` `SnapDiff::LegacyShims::CONFIG_MAPPING`. + +### Act now: what 2.1 removes + +2.1 deletes the v1 namespace trees, the ChunkyPNG driver, `shift_distance_limit`, +and the whole driver abstraction — **libvips becomes the only backend**. Removing +public API in a minor is a deliberate departure from strict semver; publishing the +contract one release ahead is the mitigation. 2.0 warns once per process for each of: + +| You will hear about it when you… | Do this in 2.0 | +|---|---| +| select `driver: :chunky_png` | add `gem "ruby-vips"` and drop the option | +| run `driver: :auto` **without ruby-vips** — nothing in your setup says `chunky_png`, so this warning is the only sign 2.1 will break the process | install libvips + `ruby-vips` | +| set `shift_distance_limit` | use `median_filter_window_size`, `tolerance` or `color_distance_limit` | +| read `SnapDiff::Drivers.loaded` / `.available` | require `ruby-vips` instead of branching on a detected list | +| `include SnapDiff::Driver` in your own driver | nothing — custom drivers have no migration path | + +Two removals 2.0 cannot warn about, so they are written down instead: **`driver:` +as a setting goes away entirely** — `SnapDiff.config.driver = :vips` and +`Capybara::Screenshot::Diff.driver = :vips` raise `NoMethodError` on 2.1, and the +per-screenshot `screenshot "x", driver: :vips` is silently ignored there. Delete +both; one backend needs no selection. The legacy `LOADED_DRIVERS` / +`AVAILABLE_DRIVERS` constants are also plain aliases with nothing to hook. + +### Added +- **`SnapDiff` is the canonical namespace** — the implementation lives in + `lib/snap_diff/`. Configuration (`SnapDiff::Config`, one storage behind every + settings surface), errors (`SnapDiff::Error` and friends), `SnapDiff::Region`, + `SnapDiff::Reporters::Default`, `SnapDiff.session`, `SnapDiff::Reporting.register`, + and integration requires under `snap_diff/integrations/…`. Start here for new + code: [docs/snapdiff.md](docs/snapdiff.md) +- **One consolidated config object** — all 27 settings on `SnapDiff.config`, via + `SnapDiff.configure { |config| … }`. Old and new surfaces share one storage, so a + write through either is visible through the other +- **`SnapDiff::Error` is the base class for every error the gem defines** — + `ExpectationNotMet`, `UnstableImage`, `WindowSizeMismatchError` and + `DualInstallError` all inherit it, so one `rescue SnapDiff::Error` covers them. + (Misuse still surfaces as plain Ruby: `ArgumentError` for bad arguments, + `RuntimeError` when no image backend is installed.) +- **Deprecation warnings name your call site**, so migration is warning-driven + rather than grep-driven +- **Dual-install guard** — installing both `capybara-screenshot-diff` and + `snap_diff-capybara` raises `SnapDiff::DualInstallError` at require time instead of + silently loading files from whichever gem activated first +- **The gem is also published as + [`snap_diff-capybara`](https://rubygems.org/gems/snap_diff-capybara)** — identical + content and version, matching this repository. `capybara-screenshot-diff` remains + the name to install; the mirror reserves the forward-looking one. **Install one, + never both.** +- New documentation shipped inside the gem: + [docs/snapdiff.md](docs/snapdiff.md) (canonical API) and + [docs/UPGRADING.md](docs/UPGRADING.md) (every renamed constant, which names warn, + rollback) + +### Changed +- **No `activesupport` at runtime.** 1.x required `active_support/core_ext/…` + without declaring the dependency, so a non-Rails install could fail to load. 2.0 + requires nothing beyond `capybara` +- The images-holder struct is now `SnapDiff::Comparison::Images`, ending the + two-classes-one-name collision with the comparator +- The packaged gem is an explicit allow-list — `lib/`, `docs/`, `README.md`, + `LICENSE.txt`, `CHANGELOG.md`. 1.x shipped `Rakefile`, `gems.rb` and the gemspec, + and omitted the README + +### Fixed +- Annotation color constants resolve under a bare `require "snap_diff"`; a differing + comparison previously raised `NameError` there +- `require "snap_diff/integrations/…"` loads the full `SnapDiff` surface, and + `gem "snap_diff-capybara"` works with `Bundler.require` +- **Failure messages no longer dump a libvips pointer struct.** The comparison + metadata carried the raw `diff_mask` image into the error text + (`"diff_mask":{"ptr":{}…}`); it is excluded now, leaving the metrics +- Reporter failure warnings use one brand and name the failing reporter class + +### Known limitations +- **Fork-based parallel tests produce no HTML report.** Under Minitest's forked + parallel executor (`parallelize(workers: N)`, the Rails default), each worker + accumulates its assertions in its own process, while the report is written from + `Minitest.after_run` in the parent — which never sees them. Pass/fail is correct + and the diff image artifacts are still written; only the HTML report is missing. + Fixed in 2.1 + +### Unchanged +- Ruby 3.2+, Capybara `>= 2, < 4`, the `screenshot` / `assert_matches_screenshot` + DSL, every capture and comparison option, baseline file names and formats + +--- + ## [v2.0.0.beta3] - 2026-08-23 Fixes the canonical `SnapDiff` entry points, which were incomplete in beta2. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 581f276e..dc658049 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -152,17 +152,15 @@ Include: To release a new version: -1. Update the version number in [lib/capybara/screenshot/diff/version.rb](lib/capybara/screenshot/diff/version.rb) +1. Update the version number in [lib/snap_diff/version.rb](lib/snap_diff/version.rb) — the + only place it lives; the gemspec and the legacy + `lib/capybara/screenshot/diff/version.rb` both read it 2. Update [CHANGELOG.md](CHANGELOG.md) with the new version and date -3. Create a GitHub Release: - - Go to [Actions → Release](https://github.com/snap-diff/snap_diff-capybara/actions/workflows/release.yml) - - Click **Run workflow**, enter the version number - - The workflow will: test → tag → publish to RubyGems → create GitHub Release - -Or manually: - -```bash -bundle exec rake release -``` - -This creates a git tag, pushes commits and tags, and pushes the `.gem` file to [rubygems.org](https://rubygems.org). +3. Go to [Actions → Release](https://github.com/snap-diff/snap_diff-capybara/actions/workflows/release.yml), + click **Run workflow** and enter the version number. The workflow verifies the version + against `lib/`, tests, tags, publishes **both** gem names, and creates the GitHub Release. + +**Do not use `rake release`.** It is inherited from `bundler/gem_tasks` and publishes only +`capybara-screenshot-diff`, skipping the `snap_diff-capybara` mirror — the two gems must +never diverge in version. The full runbook, including the trusted-publishing prerequisites +and what to do when a run fails halfway, is in [docs/RELEASE_PREP.md](docs/RELEASE_PREP.md). diff --git a/README.md b/README.md index 909e337a..28e94b4e 100644 --- a/README.md +++ b/README.md @@ -11,9 +11,11 @@ Stop shipping UI bugs. Take screenshots in your Capybara tests, commit baselines **Why this gem?** Baselines live in git — review UI changes in pull requests like you review code. Runs offline, works in CI, zero vendor lock-in. Unlike Percy/Chromatic (paid SaaS), nothing to sign up for. Unlike BackstopJS, no Node required. -> **2.0 experiment (beta):** the gem is moving to a `SnapDiff` canonical namespace. Opt in with `gem "capybara-screenshot-diff", "2.0.0.beta3"` (or the latest 2.0.0 prerelease; prereleases are never installed by default — normal installs stay on 1.x). Legacy names keep working; the first legacy API a process touches prints one migration notice (lazily shimmed constants also warn once each — see [which names warn](docs/UPGRADING.md#deprecation-warnings)), silenceable via `SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`. Writing new code? Start from [SnapDiff — the canonical API](docs/snapdiff.md), which uses canonical names only. Migrating an existing suite? See the [upgrade guide](docs/UPGRADING.md). Share feedback on [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166). +> **2.0 is the transitional release.** The gem's canonical namespace is now `SnapDiff`. Upgrading from 1.x is a version bump — every legacy `Capybara::Screenshot::Diff` / `CapybaraScreenshotDiff` name still resolves to the same object and keeps working. A legacy **config accessor**, an `include Capybara::Screenshot[::Diff]`, `Diff.default_options`, or a lazily shimmed legacy constant prints one migration notice per process (shimmed constants also warn once each). **The legacy integration require is not one of those doors** — `require "capybara_screenshot_diff/minitest"` plus `include CapybaraScreenshotDiff::Minitest::Assertions` is silent by design, because those names are eager aliases with no `const_missing` to hook. See [which names warn](docs/UPGRADING.md#deprecation-warnings). Silence the ones that do via `SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`. > -> Starting with the 2.0 prereleases the gem is also published as [`snap_diff-capybara`](https://rubygems.org/gems/snap_diff-capybara) — identical content and versions under the forward-looking name, matching this repository. Install either; don't install both. +> **2.1 removes what 2.0 warns about**: the legacy namespaces, the ChunkyPNG driver, `shift_distance_limit`, the `driver:` setting and the driver abstraction — libvips becomes the only backend. There is no 3.0. Writing new code? Start from [SnapDiff — the canonical API](docs/snapdiff.md), which uses canonical names only. Migrating an existing suite? See the [upgrade guide](docs/UPGRADING.md). +> +> **Two gem names, one gem — install `capybara-screenshot-diff`.** From 2.0.0 on, the identical content is also published as [`snap_diff-capybara`](https://rubygems.org/gems/snap_diff-capybara), the forward-looking name matching this repository. Do not reach for it yet: that name's only non-prerelease before 2.0.0 is a `0.0.1` placeholder containing a README and no Ruby files, so an unpinned `gem "snap_diff-capybara"` installs an empty gem and fails with `LoadError`. **Always pin the version**, and **install one name, never both** — with both in a Gemfile the gem raises `SnapDiff::DualInstallError` at require time. ## Quick Start (5 minutes) @@ -21,24 +23,38 @@ Stop shipping UI bugs. Take screenshots in your Capybara tests, commit baselines ```ruby # Gemfile -gem 'capybara-screenshot-diff' -gem 'ruby-vips' # Optional: 10x faster comparisons +gem 'capybara-screenshot-diff', '2.0.0.beta3' # current 2.0 prerelease; 2.0.0 final is not out yet +gem 'ruby-vips' # The image backend. Needs libvips — see Installation below ``` +Pin the exact prerelease. Bundler never resolves a prerelease from a plain requirement, so +`'~> 2.0'` fails with `Could not find gem 'capybara-screenshot-diff (~> 2.0)'` until 2.0.0 +ships. Once it does, `'~> 2.0'` is the pin to use. + +The gem ships no image backend of its own. Add `ruby-vips` (recommended, and the only +backend from 2.1 on) or `chunky_png` (pure Ruby, no system library, removed in 2.1) — with +neither, comparisons raise `Wrong adapter nil. Available adapters: []`. + ```ruby # test/test_helper.rb -require 'capybara_screenshot_diff/minitest' +require "snap_diff/integrations/minitest" ``` ```ruby # test/application_system_test_case.rb +require "test_helper" + class ApplicationSystemTestCase < ActionDispatch::SystemTestCase - include CapybaraScreenshotDiff::Minitest::Assertions + driven_by :selenium, using: :headless_chrome, screen_size: [1400, 1400] + + include SnapDiff::Minitest::Assertions end ``` ```ruby # test/system/homepage_test.rb +require "application_system_test_case" + class HomepageTest < ApplicationSystemTestCase test "homepage" do visit "/" @@ -47,22 +63,40 @@ class HomepageTest < ApplicationSystemTestCase end ``` +> **Pin the browser.** `driven_by` is not optional decoration in a pixel-diffing suite. +> Rails falls back to a *visible* browser at whatever size and pixel ratio the machine +> gives it: the same page that captures as **1400x1257** with the line above captures as +> **2800x1610** without it, and every comparison then fails with `Dimensions have changed`. +> Both `require` lines matter too — Rails does not autoload `test/`, so dropping either +> one raises `NameError: uninitialized constant`. + +`SnapDiff::Minitest::Assertions` already includes `SnapDiff::DSL`, so no separate include is +needed. The legacy `require "capybara_screenshot_diff/minitest"` + +`include CapybaraScreenshotDiff::Minitest::Assertions` still work and resolve to these same +objects — 2.1 removes them, so new suites should start here. See +[SnapDiff — the canonical API](docs/snapdiff.md). + (`screenshot` still works as a shorthand, and is safe to override in your own helpers — the gem no longer calls it internally.) Then run these steps in order: ```bash # Step 1: Save baselines (first run always passes) -bundle exec rake test +bin/rails test:system # Step 2: Commit baselines to git git add doc/screenshots/ git commit -m "chore: add screenshot baselines" # Step 3: Now comparisons work — change your UI and re-run -bundle exec rake test +bin/rails test:system ``` +> **Run the task that actually runs system tests.** In a Rails app, `rake test` +> and `rails test` skip `test/system/` — you get `0 runs` and no baselines, which +> looks like a pass. Use `rails test:system` (or `rails test test/system`). +> Outside Rails, run whatever task loads your Capybara tests. + After Step 1, you'll see: ```text doc/screenshots/ @@ -81,13 +115,14 @@ snap_diff_report.html If you skip Step 2 and push to CI, the build will fail — `fail_if_new` is `true` by default in CI. -For RSpec, Cucumber, or non-Rails setup, see [Framework Setup](docs/framework-setup.md). +For RSpec, Cucumber, or non-Rails setup, see [SnapDiff — the canonical API](docs/snapdiff.md#quick-start) +(or [Framework Setup](docs/framework-setup.md) for the same wiring in legacy names). ### For Non-Rails Projects (Hugo, Jekyll, Static Sites) ```ruby -require 'capybara_screenshot_diff/static' -CapybaraScreenshotDiff.serve("_site") # or "public", "build", "dist" +require "snap_diff/static" +SnapDiff.serve("_site") # or "public", "build", "dist" ``` Then commit baselines to git just like Rails. [Full setup](docs/ci-integration.md#non-rails-projects-hugo-jekyll-static-sites). @@ -97,18 +132,59 @@ Then commit baselines to git just like Rails. [Full setup](docs/ci-integration.m The test fails with a clear message and generates diff files: ```text -Screenshot does not match for 'homepage': -({"area_size":1250,"region":[0,19,199,83],"max_color_distance":42.5}) +Screenshot does not match for 'homepage': ({"area_size":41520.0,"region":[8.0,8.0,1392.0,38.0]}) +doc/screenshots/homepage.png +doc/screenshots/homepage.base.diff.png +doc/screenshots/homepage.diff.png +doc/screenshots/homepage.heatmap.diff.png ``` -Open `doc/screenshots/homepage.diff.png` to see exactly what changed. If the change is intentional, delete the baseline and re-run to update it. +Open `doc/screenshots/homepage.diff.png` to see exactly what changed. If the change is intentional, see [Accepting an intentional change](#accepting-an-intentional-change). + +A failing run leaves five files behind — the rewritten baseline plus four artifacts: | File | Description | |------|-------------| -| `homepage.png` | Committed baseline | -| `homepage.diff.png` | Visual diff with changes highlighted in red | +| `homepage.png` | Baseline path — **rewritten** with the new capture (`git status` shows it modified) | +| `homepage.base.png` | The committed baseline, checked out of `HEAD` for the comparison | +| `homepage.diff.png` | The new capture, with changed regions highlighted | +| `homepage.base.diff.png` | The old baseline, with the same regions highlighted | | `homepage.heatmap.diff.png` | Heatmap of pixel differences | +Only `homepage.png` is committed; the `.gitignore` above keeps the other four out. + +## Accepting an intentional change + +**Baselines are read from git, not from your working directory.** Every comparison runs +`git show HEAD:` for the baseline, so a screenshot that is committed is the one you +are compared against — no matter what the file on disk says. + +That makes the obvious move the wrong one: **deleting the baseline file does nothing.** The +gem fetches the committed copy from `HEAD` and the test fails exactly as before. + +Accepting a change is therefore a **commit**, not a file operation. The run writes its new +capture to the baseline path, so `git status` shows the baseline as modified — review it and +commit it: + +```bash +git status # doc/screenshots/homepage.png is modified +git diff --stat doc/screenshots/ + +# Look at homepage.diff.png. If the change is what you wanted: +git add doc/screenshots/homepage.png +git commit -m "chore: update homepage baseline" + +bin/rails test:system # now green — HEAD holds the new baseline +``` + +> **Staging is not enough.** `git add` alone does not move `HEAD`, so a staged-but-uncommitted +> baseline is still compared against the old committed one. You cannot get a green local run +> until you commit. That is by design: the baseline under review in a pull request is exactly +> the baseline the suite uses. + +Reviewing the change is what the pull request is for — the updated `.png` shows up as an image +diff next to the code that caused it. + ## Web UI for Reviewing Screenshot Changes Add one line to get an interactive dashboard for reviewing all screenshot differences: @@ -168,7 +244,7 @@ Yes. First run saves baselines and always passes. Run tests again to compare aga
How do I update baselines after intentional UI changes? -Delete the baseline file and re-run tests: `rm doc/screenshots/homepage.png && bundle exec rake test`. Or update all: `rm -rf doc/screenshots/ && bundle exec rake test`. +**Not by deleting the file** — baselines are read from git (`git show HEAD:`), so `rm` has no effect on what you are compared against. Commit the new capture instead: `git add doc/screenshots/homepage.png && git commit`. See [Accepting an intentional change](#accepting-an-intentional-change).
@@ -186,22 +262,29 @@ Set `window_size` for consistent dimensions and use `perceptual_threshold: 2.0`
Will this slow down my tests? -Comparisons add ~50ms per image with VIPS. Without `ruby-vips`, ChunkyPNG is used (slower but no system dependency). `stability_time_limit` adds wait time — keep it low (0.1-0.5s) or use `disable_animations` instead. +Comparisons add ~50ms per image with VIPS. If you add `chunky_png` to your Gemfile instead, it is used as a pure-Ruby fallback (slower, no system dependency, and removed in 2.1). `stability_time_limit` adds wait time — keep it low (0.1-0.5s) or use `disable_animations` instead.
Debug mode -`DEBUG=1 bundle exec rake test` keeps `.diff.png` files for inspection. +You do not need a flag to keep the diff images — a failing run leaves `.diff.png`, +`.base.diff.png`, `.heatmap.diff.png` and `.base.png` on disk and nothing in the gem +deletes them (`SnapManager#cleanup!` is this repository's own test-harness call, not +something your suite runs). + +`DEBUG=1` does one thing: it makes the HTML reporter print why it skipped an assertion +instead of failing quietly — useful when `snap_diff_report.html` is missing entries.
## Installation -**Requirements:** Ruby 3.2+. Rails 7.1+ for Rails integration; non-Rails projects supported via `CapybaraScreenshotDiff.serve()`. For the `:vips` driver: [libvips 8.9+](https://libvips.github.io/libvips/install.html). On macOS: `brew install vips`. On Ubuntu: `apt-get install libvips-dev`. +**Requirements:** Ruby 3.2+, Capybara 2–3. Rails 7.1+ for Rails integration; non-Rails projects supported via `SnapDiff.serve()`. For the `:vips` driver (recommended, and the only backend from 2.1 on): [libvips 8.9+](https://libvips.github.io/libvips/install.html). On macOS: `brew install vips`. On Ubuntu: `apt-get install libvips-dev`. ## Docs - [SnapDiff — the canonical API](docs/snapdiff.md) — setup, config, object map, custom drivers & reporters, canonical names only +- [Upgrading](docs/UPGRADING.md) — 1.x → 2.0, every renamed constant, which names warn, what 2.1 removes, rollback - [Framework Setup](docs/framework-setup.md) — Minitest, RSpec, Cucumber - [CI & Non-Rails Integration](docs/ci-integration.md) — GitHub Actions, reusable action, static sites, baseline updates - [Configuration Reference](docs/configuration.md) — all options explained @@ -211,11 +294,11 @@ Comparisons add ~50ms per image with VIPS. Without `ruby-vips`, ChunkyPNG is use ## Development -After checking out the repo, run `bin/setup` then `rake test`. See [Docker Testing](docs/docker-testing.md) for reproducible CI-matching test runs. +After checking out the repo, run `bin/setup` then `rake test`. See [Docker Testing](https://github.com/snap-diff/snap_diff-capybara/blob/master/docs/docker-testing.md) for reproducible CI-matching test runs. ## Contributing -See [CONTRIBUTING.md](CONTRIBUTING.md) +See [CONTRIBUTING.md](https://github.com/snap-diff/snap_diff-capybara/blob/master/CONTRIBUTING.md) ## License diff --git a/Rakefile b/Rakefile index c0c705ed..3562df55 100644 --- a/Rakefile +++ b/Rakefile @@ -5,13 +5,13 @@ require "rake/testtask" task default: :test -# THE 3.0 SPLIT. +# THE 2.1 SPLIT. # # test/legacy/ holds every test whose SUBJECT is the v1 compatibility surface # -- the old Capybara::Screenshot / CapybaraScreenshotDiff namespaces, their # deprecation warnings, and the gates that keep lib/capybara* alias-only. # Those tests guard the v1 contract for the whole 2.x line, so they stay and -# stay green; in 3.0 they are deleted by the same commit that deletes what +# stay green; in 2.1 they are deleted by the same commit that deletes what # they test: # # git rm -r lib/capybara* lib/capybara_screenshot_diff.rb \ @@ -23,7 +23,7 @@ task default: :test # # `rake test` -- everything, today's gate. # `rake test:canonical` -- exactly what must still pass once test/legacy and -# the v1 trees are gone. THE 3.0 GATE. +# the v1 trees are gone. THE 2.1 GATE. # `rake test:unit` -- unit-sized tests; test/legacy is unit-sized too # (legacy/ marks lifetime, not kind), so it is in. LEGACY_SURFACE_TESTS = "test/legacy/**/*_test.rb" @@ -34,7 +34,7 @@ Rake::TestTask.new(:test) do |t| t.test_files = FileList["test/**/*_test.rb"] end -desc "Run every test that must survive the 3.0 deletion of the v1 surface" +desc "Run every test that must survive the 2.1 deletion of the v1 surface" Rake::TestTask.new("test:canonical") do |t| t.libs << "test" t.libs << "lib" diff --git a/capybara-screenshot-diff.gemspec b/capybara-screenshot-diff.gemspec index fea3dbc2..01853679 100644 --- a/capybara-screenshot-diff.gemspec +++ b/capybara-screenshot-diff.gemspec @@ -9,20 +9,28 @@ Gem::Specification.new do |spec| spec.version = SnapDiff::VERSION spec.authors = ["Uwe Kubosch"] spec.email = ["uwe@kubosch.no"] - spec.summary = "Track your GUI changes with diff assertions" - spec.description = "Save screen shots and track changes with graphical diff" + spec.summary = "Visual regression testing for Capybara — screenshot diffs in your test suite" + spec.description = "Take screenshots in your Capybara tests, commit the baselines to git, " \ + "and let your suite fail on unintended visual changes. Runs offline, no SaaS." spec.homepage = "https://github.com/snap-diff/snap_diff-capybara" spec.required_ruby_version = ">= 3.2" spec.license = "MIT" spec.metadata["allowed_push_host"] = "https://rubygems.org/" + spec.metadata["source_code_uri"] = spec.homepage + spec.metadata["changelog_uri"] = "#{spec.homepage}/blob/master/CHANGELOG.md" + spec.metadata["bug_tracker_uri"] = "#{spec.homepage}/issues" + spec.metadata["documentation_uri"] = "#{spec.homepage}/blob/master/docs/snapdiff.md" + spec.metadata["rubygems_mfa_required"] = "true" # Allow-list: everything a consumer needs at runtime plus the shipped docs. # Build/dev files (gems.rb, Rakefile, the gemspec itself, tests, CI) stay out. spec.files = `git ls-files -z`.split("\x0") .grep(%r{\A(lib/|docs/|README\.md\z|LICENSE\.txt\z|CHANGELOG\.md\z)}) - .grep_v(%r{\Adocs/RELEASE_PREP\.md\z}) # maintainer-only, not user documentation + # Contributor docs: they describe releasing this gem and running its own suite + # via bin/dtest, which is not packaged. Nothing a consumer can act on. + .grep_v(%r{\Adocs/(RELEASE_PREP|docker-testing)\.md\z}) - spec.bindir = "exe" - spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) } + # No executables: the allow-list above never matches exe/, so bindir and + # executables would always be empty. spec.require_paths = ["lib"] spec.add_development_dependency "actionpack", ">= 7.1", "< 9" diff --git a/docs/RELEASE_PREP.md b/docs/RELEASE_PREP.md index a50b1ddc..a21101c2 100644 --- a/docs/RELEASE_PREP.md +++ b/docs/RELEASE_PREP.md @@ -1,44 +1,90 @@ -# Release Preparation — v1.15.1 +# Release runbook -## Summary +Maintainer-only. This file is deliberately excluded from the packaged gem. -HIGH-severity VipsDriver resize fix (retina halving could enlarge screenshots -and corrupt baselines) plus the driver contract tests that caught it. +Releases are **one `workflow_dispatch`**. There is no `rake release`, no manual +`gem push`, and no local credential. Everything below either happens in CI or has to +be committed to `master` before you dispatch. -## Release Checklist +## What the Release workflow does -### Pre-Release +[`.github/workflows/release.yml`](../.github/workflows/release.yml), dispatched from +[Actions → Release](https://github.com/snap-diff/snap_diff-capybara/actions/workflows/release.yml) +with a single `version` input (e.g. `2.0.0`, or `2.1.0.beta1`): -- [x] Update version to `1.15.1` -- [x] Run tests: `bundle exec rake test:unit` (376 runs, 0 failures) -- [x] Update CHANGELOG.md +1. **Verify version** — reads `Capybara::Screenshot::Diff::VERSION` out of `lib/` and + fails if it does not equal the input. This is the guard that makes the dispatch + safe: the version lives in `lib/snap_diff/version.rb` and must already be on + `master`. +2. **Test** — `bundle exec rake test:unit` on Ruby 4.0. +3. **Tag** — creates and pushes `v`. Idempotent: it skips if the tag already + exists at HEAD and fails loudly if it exists at a *different* commit. A failed run + can be re-dispatched without cleanup. +4. **Publish `capybara-screenshot-diff`** — `rubygems/release-gem@v1`, via RubyGems + trusted publishing (OIDC). No API key is stored anywhere. +5. **Publish the `snap_diff-capybara` mirror** — the same gemspec is loaded, renamed + in memory, built and pushed. The mirror gemspec is *generated in CI, never + committed*, so local `gem build` and the `gemspec` directive in `gems.rb` stay + unambiguous. It reuses the credential `release-gem` already set up, which covers + any gem whose rubygems.org settings trust this repo + workflow. +6. **GitHub Release** — body links to the CHANGELOG and upgrade guide **at the tag**. + Prerelease status is auto-detected from the tag, so `v2.1.0.beta1` lands as + *Pre-release* and never displaces the *Latest* badge. -### Release (One Click) +### Prerequisites that live outside this repo -1. Push to GitHub -2. Go to [Actions → Release](https://github.com/snap-diff/snap_diff-capybara/actions/workflows/release.yml) -3. Click **Run workflow**, enter `1.15.1` -4. Workflow will: test → tag → publish to RubyGems → create GitHub Release +- **Both** gem names must trust this repo + `release.yml` as a trusted publisher on + rubygems.org — `capybara-screenshot-diff` **and** `snap_diff-capybara`. If only one + does, step 4 or 5 fails after the tag is already pushed; re-dispatch after fixing. +- If tag protection rules are ever added to this repo, `v*` must allow + `github-actions[bot]` to push — that is how step 3 creates the tag. -### Post-Release +## Before you dispatch -- [ ] Verify on [RubyGems](https://rubygems.org/gems/capybara-screenshot-diff) -- [ ] Verify GitHub Release created +- [ ] `lib/snap_diff/version.rb` bumped to the exact version you will type into the + workflow. Nothing else holds a version — the gemspec, the legacy + `capybara/screenshot/diff/version.rb` and the mirror gemspec all read it. +- [ ] `CHANGELOG.md` has a section for this version with a real date (not + `unreleased`), written for someone upgrading from the last **stable** release + rather than from the previous prerelease. +- [ ] Docs carry no stale version pins — `README.md`, `docs/UPGRADING.md`, + `docs/snapdiff.md`. Grep for the previous version string. +- [ ] **Releasing 2.0.0: swap every install snippet from the `2.0.0.beta3` pin to + `"~> 2.0"`,** and delete the "final is not out yet" clauses next to them. Until + 2.0.0 exists on rubygems, `gem "capybara-screenshot-diff", "~> 2.0"` hard-fails + `bundle install` (Bundler never resolves a prerelease from a plain requirement), so + the good pin has to land *with* the release, not before it. The snippets live in + `README.md`, `CHANGELOG.md`, `docs/UPGRADING.md` (×2) and `docs/migration-guide.md` — + re-grep rather than trusting that list: + `grep -rn 'capybara-screenshot-diff.\{0,4\}2\.0' README.md CHANGELOG.md docs/` +- [ ] `mise x ruby@4.0.6 -- bundle exec rake test` (full suite, both gates) and + `mise x ruby@4.0.6 -- bundle exec standardrb` are green. +- [ ] CI is green on `master` at the commit you are releasing — the workflow only + runs `test:unit`, which is a subset. +- [ ] `gem build capybara-screenshot-diff.gemspec` and inspect the file list if + anything touched the gemspec allow-list. `*.gem` is gitignored, but delete the + artifact anyway — a stale one in the working tree is confusing. -## What Changed +### Prereleases -### Added -- `SnapDiff::Config` — flat, additive consolidation of all 27 settings - (`SnapDiff.config` / `SnapDiff.configure`); old accessors stay canonical +Nothing special to configure. Use a prerelease version string (`2.1.0.beta1`) in +`version.rb` and in the dispatch input. RubyGems never resolves a prerelease by +default, and GitHub marks the release *Pre-release* on its own. -### Fixed -- `pending_if_new` no longer converts real teardown/after-hook failures into - pending tests (Minitest defers to `after_teardown`; RSpec uses `append_after`; - known residual for consumer `append_after` hooks documented) -- `BacktraceFilter` custom `lib_directory` matches on a path boundary +## After -### Internal -- Guard tests for failure masking and skip-area/VCS-baseline regressions; - two files merged into sole consumers (constant paths preserved) +- [ ] Both gems visible and at the same version: + [capybara-screenshot-diff](https://rubygems.org/gems/capybara-screenshot-diff), + [snap_diff-capybara](https://rubygems.org/gems/snap_diff-capybara). A version + published under only one name is the failure mode to watch for — the two gems + ship identical files and the dual-install guard assumes they never diverge. +- [ ] GitHub Release created, with the right Latest/Pre-release status. +- [ ] `gem install capybara-screenshot-diff -v ` in a scratch dir resolves. -See [CHANGELOG.md](../CHANGELOG.md) for full details. +## If a run fails halfway + +Re-dispatch the same version. The tag step is idempotent and `gem push` rejects a +duplicate version, so the only real hazard is a mirror push that failed for a +credential reason — fix the trusted-publisher settings and re-dispatch. **Never** +retag: the workflow refuses to move an existing tag and you should not do it by hand +either. diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index e7b623e0..41f54eab 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -1,12 +1,12 @@ # Upgrading -## Upgrading to v2.0 (alpha) +## Upgrading to v2.0 ### Overview Version 2.0 introduces a new canonical namespace (`SnapDiff`) for cleaner, more discoverable code. The public DSL remains unchanged — your existing `screenshot` and `assert_matches_screenshot` calls work without modification. This guide covers the optional migration path for settings and the new namespace. -**Status:** `2.0.0.beta3` is an opt-in prerelease. RubyGems never installs prereleases by default resolution — normal `bundle update` keeps you on the 1.x line. The final 2.0.0 ships only after adopter feedback; please report anything surprising on [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166). +**Status:** 2.0 is the **transitional** release — the v1 API and the canonical `SnapDiff` API both work. **2.1 removes** everything 2.0 warns about (the legacy namespaces, the ChunkyPNG driver, `shift_distance_limit`, the `driver:` setting and the driver abstraction). There is no 3.0. Migrating on 2.0 is optional; doing it before 2.1 is not. **Estimated upgrade time:** 5–15 minutes (most users need only the Gemfile pin) @@ -14,22 +14,37 @@ Version 2.0 introduces a new canonical namespace (`SnapDiff`) for cleaner, more [SnapDiff — the canonical API](snapdiff.md): the same setup, configuration, and extension points with canonical names only, no legacy shapes to unlearn. -**Breaking changes:** None for the DSL; one migration notice per process plus a deprecation warning per legacy constant you reference (both suppressible), plus two known alpha caveats (see below) +**Breaking changes:** None for the DSL; one migration notice per process plus a deprecation warning per legacy constant you reference (both suppressible), plus the [known caveats](#known-caveats) below --- ### The Short Version (Most Users) ```ruby -# In your Gemfile — the exact prerelease version is required to opt in -gem "capybara-screenshot-diff", "2.0.0.beta3" # or the latest 2.0.0 prerelease +# In your Gemfile +gem "capybara-screenshot-diff", "2.0.0.beta3" # current 2.0 prerelease; 2.0.0 final is not out yet ``` +**Pin the exact prerelease.** An unpinned `gem "capybara-screenshot-diff"` resolves to the 1.x +line, and `"~> 2.0"` does not resolve at all — Bundler never picks a prerelease from a plain +requirement, so it fails with `Could not find gem 'capybara-screenshot-diff (~> 2.0)'`. Once +2.0.0 ships, `"~> 2.0"` is the pin to use. + +The same content is also published as `snap_diff-capybara` from 2.0.0 on, but that name's +only earlier non-prerelease is a `0.0.1` placeholder with no Ruby files in it — unpinned, it +installs an empty gem and raises `LoadError`. Stay on `capybara-screenshot-diff`, and install +**one** name: with both in a Gemfile the gem raises `SnapDiff::DualInstallError` at require +time. + ```bash bundle install -bundle exec rake test +bin/rails test:system # the task that actually runs your screenshot tests ``` +> **Not `rake test` / `rails test`.** In a Rails app those skip `test/system/`, so you get +> `0 runs, 0 assertions, 0 failures` — a clean-looking pass that compared nothing. Outside +> Rails, run whatever task loads your Capybara tests. + **That's it.** Your existing code works unchanged. The old namespaces (`Capybara::Screenshot::Diff`, `CapybaraScreenshotDiff`) are shimmed with deprecation warnings; the new one (`SnapDiff`) is available if you want to modernize. --- @@ -45,6 +60,7 @@ The implementation now lives in `lib/snap_diff/` under the `SnapDiff` namespace. | `Capybara::Screenshot::Diff::ImageCompare` | `SnapDiff::Comparison` | | `Capybara::Screenshot::Diff::Difference` | `SnapDiff::ComparisonResult` | | `Capybara::Screenshot::Diff::Drivers::BaseDriver` | `SnapDiff::Driver` (now a mixin — see below) | +| `Capybara::Screenshot::Os` | `SnapDiff::Os` | | `CapybaraScreenshotDiff::SnapManager` / `::Snap` | `SnapDiff::SnapManager` / `SnapDiff::Snap` | | `CapybaraScreenshotDiff::RED_RGBA` / `::ORANGE_RGBA` | `SnapDiff::RED_RGBA` / `SnapDiff::ORANGE_RGBA` | | `CapybaraScreenshotDiff::Minitest::Assertions` | `SnapDiff::Minitest::Assertions` | @@ -114,7 +130,6 @@ Capybara::Screenshot::Diff.configure do |screenshot, diff| screenshot.window_size = [1280, 1024] screenshot.blur_active_element = false diff.tolerance = 0.0005 - diff.driver = :vips end ``` @@ -125,7 +140,6 @@ SnapDiff.start do |screenshot, diff| screenshot.window_size = [1280, 1024] screenshot.blur_active_element = false diff.tolerance = 0.0005 - diff.driver = :vips end ``` @@ -136,7 +150,6 @@ SnapDiff.configure do |config| config.window_size = [1280, 1024] config.blur_active_element = false config.tolerance = 0.0005 - config.driver = :vips end ``` @@ -215,6 +228,19 @@ warns once per process per subject, through the same channel and the same silenc | read `SnapDiff::Drivers.available` | driver detection | require `ruby-vips` instead of branching on a detected list | | `include SnapDiff::Driver` in your own driver class | the driver mixin | nothing — see below | +> **The one removal on this list that 2.0 cannot warn you about: the `driver:` setting +> itself.** `SnapDiff.config.driver = :vips` and the legacy +> `Capybara::Screenshot::Diff.driver = :vips` are **silent** in 2.0 and raise +> `NoMethodError: undefined method 'driver='` in 2.1, at config time before any test runs. +> The per-screenshot form — `screenshot "index", driver: :vips` — is silent in 2.0 **and** +> in 2.1: per-screenshot options are a free-form hash, so an unknown key is simply inert. +> Warning on any of this would fire on the recommended configuration, so this note is the +> warning: **delete the line, and grep for the per-screenshot one.** With libvips the only +> backend there is nothing to select, and the default just works. The same goes for +> `driver: :auto` on a machine that *has* `ruby-vips` — the `:auto` warning above only +> fires when `:auto` actually falls back to ChunkyPNG, because that is the case where 2.1 +> stops the process comparing at all. + ``` [snap_diff deprecation] `driver: :auto` selected chunky_png because libvips is not available in this process. The chunky_png driver is REMOVED in 2.1, when libvips (the `ruby-vips` gem) becomes required -- install it now, or this setup stops comparing on 2.1. See docs/drivers.md. Silence with `SnapDiff.silence_deprecations = true` or SNAP_DIFF_SILENCE_DEPRECATIONS=1. (shown once per process) (called from /app/test/test_helper.rb:12) ``` @@ -223,7 +249,7 @@ warns once per process per subject, through the same channel and the same silenc mixin, the `SnapDiff::Drivers.loaded` registry, `SnapDiff::Drivers.available` / `SnapDiff::Utils.detect_available_drivers`, and selecting a driver by name. Nothing replaces them, and this guide is not going to pretend otherwise — if you maintain a third-party driver, -say so on [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166) before 2.1 ships. +say so on [the issue tracker](https://github.com/snap-diff/snap_diff-capybara/issues) before 2.1 ships. Three spots on the same chopping block stay silent: the legacy `Capybara::Screenshot::Diff::LOADED_DRIVERS` / `::AVAILABLE_DRIVERS` aliases are plain constants @@ -276,9 +302,10 @@ export SNAP_DIFF_SILENCE_DEPRECATIONS=1 --- -### Known Alpha Caveats +### Known Caveats -Two deliberate consequences of the lazy shim design — both flagged for feedback on [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166): +Two deliberate consequences of the lazy shim design. Both go away at 2.1, when the shimmed +names go away with them: 1. **`defined?` / `const_defined?` on lazily-shimmed legacy names returns `false`/`nil`.** The shims resolve via `const_missing`, which those checks never trigger. Feature detection like `defined?(Capybara::Screenshot::Diff::ImageCompare)` must move to the `SnapDiff::` name. Everything in [Silent by design](#silent-by-design) is unaffected — those names are real constants, so `defined?`, `const_defined?` and `rescue` all behave as they always did. @@ -331,12 +358,15 @@ All settings and baselines are compatible with v1.x. Simply pin your Gemfile bac ### Summary Checklist -- [ ] Pin `gem "capybara-screenshot-diff", "2.0.0.beta3"` (or the latest 2.0.0 prerelease) in your Gemfile +- [ ] Pin `gem "capybara-screenshot-diff", "2.0.0.beta3"` in your Gemfile (`"~> 2.0"` once 2.0.0 ships) - [ ] Run `bundle install` -- [ ] Run your test suite to verify no regressions -- [ ] (Optional) Migrate config to the `SnapDiff` namespace +- [ ] Run your system tests (`bin/rails test:system`, not `rake test`) to verify no regressions +- [ ] Read the warnings it prints — each one names something 2.1 removes +- [ ] Add `gem "ruby-vips"` if you are not already on it (2.1 makes libvips the only backend) +- [ ] Drop `driver:` from your config — it is silent in 2.0 and gone in 2.1 +- [ ] (Optional, but do it before 2.1) Migrate config and constants to the `SnapDiff` namespace - [ ] (Optional) Silence deprecation warnings if not ready to migrate -- [ ] Report anything surprising on [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166) +- [ ] Report anything surprising on [the issue tracker](https://github.com/snap-diff/snap_diff-capybara/issues) --- @@ -359,7 +389,7 @@ gem 'capybara-screenshot-diff', '~> 1.13.0' ```bash bundle update capybara-screenshot-diff -bundle exec rake test +bin/rails test:system # `rake test` skips test/system/ — 0 runs, no comparison ``` **That's it!** Existing `screenshot` calls work unchanged. New methods available if needed. @@ -444,7 +474,7 @@ gem 'capybara-screenshot-diff', '~> 1.12.0' ```bash bundle update capybara-screenshot-diff -bundle exec rake test # Verify tests still pass +bin/rails test:system # Verify tests still pass (`rake test` skips test/system/) ``` **That's it!** The zero-config setup still works out of the box. Your existing screenshot comparisons will continue to work with v1.12.0. @@ -649,7 +679,6 @@ Use the new `Diff.configure` block: Capybara::Screenshot::Diff.configure do |screenshot, diff| screenshot.window_size = [1280, 1024] screenshot.stability_time_limit = 1 - diff.driver = :vips diff.tolerance = 0.0005 end ``` @@ -700,7 +729,7 @@ bundle update capybara-screenshot-diff ### Step 3: Run Tests ```bash -bundle exec rake test +bin/rails test:system # `rake test` skips test/system/ — 0 runs, no comparison ``` ### Step 4: Check for New Screenshot Failures @@ -754,13 +783,14 @@ Capybara::Screenshot.hide_caret = false Then re-record baselines with the new defaults (recommended): ```bash -# Delete old baselines -rm doc/screenshots/*.png +# Do NOT delete the baselines — they are read from git (`git show HEAD:`), so +# removing the files changes nothing. Re-recording is a commit. -# Run tests to generate new baselines -bundle exec rake test +# The run fails and rewrites every changed baseline in place +bin/rails test:system -# Commit new baselines +# Review the diffs, then commit the new baselines +git status git add doc/screenshots/ git commit -m "Re-record baselines with v1.12.0 defaults" ``` diff --git a/docs/architecture.md b/docs/architecture.md index dc864310..84521b1c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -127,7 +127,7 @@ Drivers abstract image processing operations. Shared default behavior lives in t **Auto-detection:** `SnapDiff::Drivers.detect_available` tries to load `:vips` first (via `ruby-vips` gem), then `:chunky_png`. The `:auto` driver mode picks the first available. `Utils.detect_available_drivers` is the older name and one-lines into it. -**Registry (ADR-008 step 5b):** `SnapDiff::Drivers.loaded` is the canonical driver-class cache — a `name => class` hash filled lazily by `Utils.find_driver_class_for`, and the registration point for custom drivers (the legacy `Capybara::Screenshot::Diff::LOADED_DRIVERS` is an eager same-object alias, so registrations through either land in the same hash). `SnapDiff::Drivers.available` is the canonical read API for the detected list, and since the 3.0-readiness pass the value lives with it, as `SnapDiff::Drivers::AVAILABLE_DRIVERS` — that constant is now the published stubbing point, and the legacy `Capybara::Screenshot::Diff::AVAILABLE_DRIVERS` is an eager same-object alias of it. `SnapDiff::Drivers.for` resolves an options hash to a driver instance. See [Custom drivers](snapdiff.md#custom-drivers). +**Registry (ADR-008 step 5b):** `SnapDiff::Drivers.loaded` is the canonical driver-class cache — a `name => class` hash filled lazily by `Utils.find_driver_class_for`, and the registration point for custom drivers (the legacy `Capybara::Screenshot::Diff::LOADED_DRIVERS` is an eager same-object alias, so registrations through either land in the same hash). `SnapDiff::Drivers.available` is the canonical read API for the detected list, and since the 2.1-readiness pass the value lives with it, as `SnapDiff::Drivers::AVAILABLE_DRIVERS` — that constant is now the published stubbing point, and the legacy `Capybara::Screenshot::Diff::AVAILABLE_DRIVERS` is an eager same-object alias of it. `SnapDiff::Drivers.for` resolves an options hash to a driver instance. See [Custom drivers](snapdiff.md#custom-drivers). ### 6. Difference Region Detection @@ -223,7 +223,7 @@ Since ADR-008 step 1 the storage ownership is inverted from the original v2 cons The legacy `Capybara::Screenshot.*` / `Capybara::Screenshot::Diff.*` accessors are thin delegators generated from `SnapDiff::LegacyShims::CONFIG_MAPPING` (both singleton and instance methods, matching what `mattr_accessor` used to define) that forward to that one object. One storage, two views — a write through either surface is visible through the other structurally, not by synchronization. -Since the 3.0-readiness pass, `lib/snap_diff/legacy_shims.rb` is the single file that holds the v1 surface as code: the `const_missing` forwarders, `CONFIG_MAPPING` and its generator, the derived forwarders (`Screenshot.active?`, `Diff.configure`, `Diff.default_options`, …) and `SnapDiff.start`. `Config` itself names nothing from the v1 namespaces — it declares its settings in `Config::SETTINGS`, and `LegacyShims::CONFIG_MAPPING` says which legacy holder each one is exposed on (an invariant pinned by `snap_diff_config_test.rb`). `lib/capybara/screenshot/diff/config_legacy.rb` remains at the old path as a pair of requires. +Since the 2.1-readiness pass, `lib/snap_diff/legacy_shims.rb` is the single file that holds the v1 surface as code: the `const_missing` forwarders, `CONFIG_MAPPING` and its generator, the derived forwarders (`Screenshot.active?`, `Diff.configure`, `Diff.default_options`, …) and `SnapDiff.start`. `Config` itself names nothing from the v1 namespaces — it declares its settings in `Config::SETTINGS`, and `LegacyShims::CONFIG_MAPPING` says which legacy holder each one is exposed on (an invariant pinned by `snap_diff_config_test.rb`). `lib/capybara/screenshot/diff/config_legacy.rb` remains at the old path as a pair of requires. The two legacy views are organized into two namespaces: diff --git a/docs/ci-integration.md b/docs/ci-integration.md index 65d0806f..ac296b4c 100644 --- a/docs/ci-integration.md +++ b/docs/ci-integration.md @@ -58,7 +58,7 @@ jobs: bundler-cache: true - name: Run tests - run: bundle exec rake test + run: bin/rails test:system # not `rake test` — it skips test/system/ - name: Upload screenshot reports if: failure() @@ -68,6 +68,10 @@ jobs: pr-comment: 'true' ``` +The three workflows on this page all run `bin/rails test:system`, because `rake test` and +`rails test` skip `test/system/` and report `0 runs` — a green CI job that compared nothing. +Not on Rails? Substitute whatever task loads your Capybara tests. + That's it. On failure, this will: - Upload diff images + HTML report as artifacts - Post a PR comment with links to the inline report and full artifact download @@ -129,7 +133,7 @@ jobs: ruby-version: '4.0' cache-apt-packages: true - - run: bundle exec rake test + - run: bin/rails test:system # not `rake test` — it skips test/system/ - uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master if: failure() @@ -160,7 +164,7 @@ jobs: run: sudo apt-get install -y libvips-dev - name: Run tests - run: bundle exec rake test + run: bin/rails test:system # not `rake test` — it skips test/system/ - name: Upload screenshot report if: failure() @@ -184,14 +188,21 @@ jobs: ## Update Baselines in CI -When intentional UI changes are made, baselines need to be re-recorded. You can do this locally: +When intentional UI changes are made, baselines need to be re-recorded. Baselines are +read from git, so accepting a change is a commit — the failing run has already written +the new capture to the baseline path: ```bash -RECORD_SCREENSHOTS=1 bundle exec rake test -git add test/fixtures/screenshots/ +bin/rails test:system # fails, and rewrites the changed baselines +git status # review what moved +git add doc/screenshots/ # the default save_path; adjust if you changed it git commit -m "chore: update screenshot baselines" ``` +With the [recommended `.gitignore`](../README.md#quick-start-5-minutes) in place, `git add +doc/screenshots/` stages only the baselines — the `.diff.png` / `.base.png` artifacts are +ignored. + Or add a workflow that maintainers can trigger manually:
@@ -225,24 +236,36 @@ jobs: ruby-version: '4.0' cache-apt-packages: true - - name: Record new baselines - run: RECORD_SCREENSHOTS=1 bundle exec rake test - continue-on-error: true + - name: Record baselines + run: bin/rails test:system + continue-on-error: true # the run fails by design; it rewrites the baselines + env: + CI: "" # see below — without this, a NEW screenshot is never written - name: Commit updated baselines run: | git config user.name "github-actions[bot]" git config user.email "github-actions[bot]@users.noreply.github.com" - git add test/fixtures/ doc/screenshots/ + git add doc/screenshots/ git diff --staged --quiet || git commit -m "chore: update screenshot baselines" git push ```
+> **`CI: ""` is what makes this job able to record a *new* baseline.** `fail_if_new` +> defaults to `true` whenever `ENV["CI"]` is set and non-empty, and the check runs +> **before** the capture ([`screenshot_matcher.rb`](https://github.com/snap-diff/snap_diff-capybara/blob/master/lib/snap_diff/screenshot_matcher.rb) +> — `check_base_screenshot` precedes `capture_screenshot`). So on a stock GitHub Actions +> runner a screenshot with no committed baseline raises `No existing screenshot found for +> …` and **nothing is written to disk** — the commit step then finds nothing to commit, +> whatever the error message suggests. Clearing `CI` for this one step (or setting +> `SnapDiff.config.fail_if_new = false`) lets both new and changed baselines land. +> *Changed* baselines are rewritten either way; only new ones need this. + **How it works:** 1. Go to Actions → "Update Screenshot Baselines" → "Run workflow" 2. Enter the branch name (e.g. your PR branch) -3. The workflow records new baselines, commits, and pushes +3. The workflow records new and changed baselines, commits, and pushes [← Back to README](../README.md) diff --git a/docs/configuration.md b/docs/configuration.md index 2406f5ed..b8af96d1 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -11,7 +11,6 @@ SnapDiff.configure do |config| config.stability_time_limit = 1 config.blur_active_element = true config.hide_caret = true - config.driver = :vips config.tolerance = 0.0005 config.color_distance_limit = 15 end @@ -26,12 +25,15 @@ Capybara::Screenshot::Diff.configure do |screenshot, diff| screenshot.stability_time_limit = 1 screenshot.blur_active_element = true screenshot.hide_caret = true - diff.driver = :vips diff.tolerance = 0.0005 diff.color_distance_limit = 15 end ``` +> **`driver:` is deliberately absent from both examples.** The setting is removed in 2.1 +> (`NoMethodError` at config time) and 2.0 cannot warn about it. Add `gem "ruby-vips"` and +> leave the selection alone — see [Drivers](drivers.md#removed-in-21-everything-on-this-page-except-vips). + `SnapDiff::Config` **is** the storage; the legacy accessors are thin delegators onto it. There is one source of truth, so a write through either surface is visible through the other — mixing them is safe, and you can migrate a suite one line at a time: diff --git a/docs/docker-testing.md b/docs/docker-testing.md index 5dd861fd..f782e0a8 100644 --- a/docs/docker-testing.md +++ b/docs/docker-testing.md @@ -13,6 +13,12 @@ This builds a Docker image with Chrome and runs the test suite against three Cap ## Recording baseline screenshots +> **This page is for contributors to this gem, not for users of it.** `bin/dtest` and +> `RECORD_SCREENSHOTS` are this repository's own test harness. `RECORD_SCREENSHOTS` is +> read by `test/test_helper.rb` — it is **not** a feature of the library and does +> nothing in your application. To accept a change in your own app, see +> [Accepting an intentional change](../README.md#accepting-an-intentional-change). + Screenshot baselines are committed to the repo and compared against during tests. When you set up the project for the first time, or after upgrading the browser/driver, you need to re-record them: ```bash diff --git a/docs/drivers.md b/docs/drivers.md index f4faede0..610e3d48 100644 --- a/docs/drivers.md +++ b/docs/drivers.md @@ -9,13 +9,16 @@ ## Removed in 2.1: everything on this page except VIPS 2.1 makes **libvips the only backend**. 2.0 is the transitional release — all of the -following still works, and warns once per process naming 2.1. Silence the warnings with +following still works. Most of it warns once per process naming 2.1; the rows marked +*silent* cannot warn, and this table is their notice. Silence the warnings with `SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`. | Removed in 2.1 | What to do in 2.0 | |---|---| | the `:chunky_png` driver | add `gem "ruby-vips"` to your Gemfile and drop `driver: :chunky_png` | -| `driver: :auto` (and the `:auto` default) | with one backend there is nothing to choose; install `ruby-vips` and the default just works | +| the `driver:` setting itself — `SnapDiff.config.driver =` and the legacy `Capybara::Screenshot::Diff.driver =` (**silent in 2.0**; on 2.1 they raise `NoMethodError` at config time) | delete the line; one backend needs no selection | +| the per-screenshot `driver:` override — `screenshot "index", driver: :vips` (**no deprecation warning in 2.0** -- but the key is validated, not inert: an unknown driver raises `RuntimeError: Wrong adapter ...`. 2.1 rejects the key outright) | delete the option, and grep for it — nothing will tell you the line is dead | +| `driver: :auto` (and the `:auto` default) — warns **only when `:auto` actually falls back to ChunkyPNG**, i.e. when `ruby-vips` is missing; **silent** otherwise | with one backend there is nothing to choose; install `ruby-vips` and the default just works | | `shift_distance_limit` | ChunkyPNG-only. Use `median_filter_window_size`, `tolerance` or `color_distance_limit` — see [Configuration](configuration.md#allowed-shift-distance) | | `SnapDiff::Driver` (the custom-driver mixin) | nothing — see below | | `SnapDiff::Drivers.loaded` (the registry) | nothing — see below | @@ -87,19 +90,20 @@ There are several options to setup active driver: `:auto`, `:chunky_png` and `:v * `:auto` - will try to load `:vips` if there is gem `ruby-vips`, in other cases will load `:chunky_png` * `:chunky_png` and `:vips` will load correspondent driver -> **2.1 keeps only `:vips`.** `:auto` and `:chunky_png` are removed; each warns once per +> **2.1 keeps only `:vips`.** `:auto` and `:chunky_png` are removed; `:chunky_png` warns once per > process in 2.0. If `:auto` is quietly running you on ChunkyPNG today (no `ruby-vips` > installed), the warning says so — that is the setup 2.1 breaks. ## Enable VIPS image processing [Vips](https://www.rubydoc.info/gems/ruby-vips/Vips/Image) driver provides a faster comparison, -and could be enabled by adding `ruby-vips` to `Gemfile`. +and is enabled by adding `ruby-vips` to your `Gemfile` (plus the libvips system package). +That is the whole setup — with `ruby-vips` installed the default `:auto` already picks it. -If need to setup explicitly Vips driver, there are several ways to do this: - -* Globally: `Capybara::Screenshot::Diff.driver = :vips` -* Per screenshot option: `screenshot 'index', driver: :vips` +**Do not select it explicitly.** Both forms are on the 2.1 removal list at the top of this page: +`Capybara::Screenshot::Diff.driver = :vips` / `SnapDiff.config.driver = :vips` raises +`NoMethodError` at config time on 2.1, and the per-screenshot `screenshot 'index', driver: :vips` +becomes inert. Neither warns in 2.0 — delete the line and grep for the per-screenshot form. With enabled VIPS there are new alternatives to process differences, which are easier to find and support. For example, `shift_distance_limit` is a very heavy operation. Instead, use `median_filter_window_size`. diff --git a/docs/framework-setup.md b/docs/framework-setup.md index 392f1f8b..d54b772b 100644 --- a/docs/framework-setup.md +++ b/docs/framework-setup.md @@ -26,12 +26,16 @@ For Minitest, need to require `capybara_screenshot_diff/minitest`. In your test class, include the `CapybaraScreenshotDiff::Minitest::Assertions` module: ```ruby +# test/application_system_test_case.rb +require 'test_helper' require 'capybara_screenshot_diff/minitest' class ApplicationSystemTestCase < ActionDispatch::SystemTestCase - # Make the Capybara & Capybara Screenshot Diff DSLs available in tests - include CapybaraScreenshotDiff::DSL - # Make `assert_*` methods behave like Minitest assertions + # Pin the browser: window size and pixel ratio are inputs to every comparison + driven_by :selenium, using: :headless_chrome, screen_size: [1400, 1400] + + # Make `assert_*` methods behave like Minitest assertions. + # This already includes CapybaraScreenshotDiff::DSL — a separate include is not needed. include CapybaraScreenshotDiff::Minitest::Assertions def test_my_feature @@ -41,6 +45,9 @@ class ApplicationSystemTestCase < ActionDispatch::SystemTestCase end ``` +Run it with `bin/rails test:system` — `rake test` / `rails test` skip `test/system/` and +report `0 runs`. + ## RSpec To use the screenshot capturing and change detection features in your tests, diff --git a/docs/migration-guide.md b/docs/migration-guide.md index 2859aedc..8172ab0e 100644 --- a/docs/migration-guide.md +++ b/docs/migration-guide.md @@ -35,15 +35,19 @@ end **After (capybara-screenshot-diff):** ```ruby -# Gemfile -gem 'capybara-screenshot-diff' +# Gemfile — pin the exact prerelease; '~> 2.0' does not resolve until 2.0.0 ships +gem 'capybara-screenshot-diff', '2.0.0.beta3' # test helper -require 'capybara_screenshot_diff/minitest' +require "snap_diff/integrations/minitest" + +# test class — test/application_system_test_case.rb +require "test_helper" -# test class class ApplicationSystemTestCase < ActionDispatch::SystemTestCase - include CapybaraScreenshotDiff::Minitest::Assertions + driven_by :selenium, using: :headless_chrome, screen_size: [1400, 1400] + + include SnapDiff::Minitest::Assertions test "homepage" do visit '/' @@ -60,7 +64,7 @@ end | First run | Uploads to Percy | Saves locally, passes automatically | | CI setup | `PERCY_TOKEN` env var | GitHub Action (3 lines) | | Diff review | Percy dashboard | `snap_diff_report.html` or PR artifacts | -| Update baselines | Percy's "Approve" button | Delete file, re-run tests, commit | +| Update baselines | Percy's "Approve" button | Re-run tests (the run rewrites the baseline), review, commit — **not** delete: baselines are read from git | | Snapshot limits | Paid plan dependent | Unlimited | | Parallel builds | Built-in | Thread-safe with t-locals + mutex | @@ -77,7 +81,7 @@ end - uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master with: ruby-version: '4.0' -- run: bundle exec rake test +- run: bin/rails test:system # not `rake test` — it skips test/system/ - uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master if: failure() with: @@ -209,7 +213,7 @@ end **After (capybara-screenshot-diff in CI):** ```yaml - uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master -- run: bundle exec rake test +- run: bin/rails test:system # not `rake test` — it skips test/system/ - uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master if: failure() with: @@ -246,9 +250,13 @@ end ### "My baselines are on Percy/Chromatic servers" -You'll need to take fresh screenshots. Either: -- Visit each page and capture manually -- Run tests with `RECORD_SCREENSHOTS=1` to generate all baselines at once +You'll need to take fresh screenshots. Run your system tests once — every screenshot +is written to its baseline path, whether or not it matched — then commit them: + +```bash +bin/rails test:system +git add doc/screenshots/ && git commit -m "chore: record baselines" +``` ### "I had hundreds of BackstopJS scenarios" @@ -262,11 +270,14 @@ screenshot 'step2' ### "My tests are slow now" -Use the VIPS driver for ~50ms comparisons per image: +Use libvips for ~50ms comparisons per image — installing the gem is the whole setup, and from +2.1 it is the only backend: ```ruby +# Gemfile gem 'ruby-vips' -Capybara::Screenshot::Diff.driver = :vips ``` +Do **not** add `driver = :vips`: that setting is removed in 2.1 and raises `NoMethodError` +there, silently, with nothing in 2.0 to warn you. See [Drivers](drivers.md). ### "The diffs look different from what I'm used to" diff --git a/docs/snapdiff.md b/docs/snapdiff.md index d431afe6..02dd9b14 100644 --- a/docs/snapdiff.md +++ b/docs/snapdiff.md @@ -5,10 +5,12 @@ reference: setup, configuration, the object map, and the extension points — al names only. The legacy `Capybara::Screenshot::Diff` / `CapybaraScreenshotDiff` names still work — they resolve -to the same objects — and the rest of the docs still teach them. The first legacy API a process -touches prints one migration notice; on top of that, *lazily shimmed* constants warn once each. -Some legacy names are silent by design. [UPGRADING.md](UPGRADING.md#deprecation-warnings) lists -exactly which is which. Nothing here replaces a working setup — it is what you write for **new** code. +to the same objects — and the rest of the docs still teach them. A legacy config accessor, +`include`, `default_options` call, or lazily shimmed constant prints one migration notice per +process; *lazily shimmed* constants warn once each on top of that. Other legacy names — the +integration requires and `CapybaraScreenshotDiff::Minitest::Assertions` among them — are silent +by design, so a quiet suite is not a migrated one. +[UPGRADING.md](UPGRADING.md#deprecation-warnings) lists exactly which is which. Nothing here replaces a working setup — it is what you write for **new** code. For migrating an existing suite, see [UPGRADING.md](UPGRADING.md). ## Quick start @@ -22,12 +24,20 @@ require "snap_diff/integrations/minitest" ```ruby # test/application_system_test_case.rb +require "test_helper" + class ApplicationSystemTestCase < ActionDispatch::SystemTestCase + # Window size and pixel ratio are inputs to every comparison — pin them. + driven_by :selenium, using: :headless_chrome, screen_size: [1400, 1400] + include SnapDiff::Minitest::Assertions # brings in SnapDiff::DSL too end ``` ```ruby +# test/system/homepage_test.rb +require "application_system_test_case" + class HomepageTest < ApplicationSystemTestCase test "homepage" do visit "/" @@ -36,6 +46,9 @@ class HomepageTest < ApplicationSystemTestCase end ``` +Run it with `bin/rails test:system`. `rake test` and `rails test` skip `test/system/` +entirely — `0 runs` and no baselines, which reads exactly like a pass. + `SnapDiff::Minitest::Assertions` already includes `SnapDiff::DSL`, so a separate `include SnapDiff::DSL` is not needed (it is harmless if you have it). @@ -104,7 +117,6 @@ All 27 settings live on one flat object, `SnapDiff.config` (a `SnapDiff::Config` SnapDiff.configure do |config| config.window_size = [1280, 1024] config.tolerance = 0.0005 - config.driver = :vips config.save_path = "doc/screenshots" end @@ -161,7 +173,7 @@ integration require; a few objects need their own require, noted below. | `SnapDiff::Region` | Bounding box value object — `from_edge_coordinates`, `to_edge_coordinates` | | `SnapDiff::DSL` | `screenshot`, `assert_matches_screenshot`, `capture_screenshot`, groups/sections | | `SnapDiff::Minitest::Assertions` | Minitest wiring (`snap_diff/integrations/minitest`) | -| `SnapDiff::Error` | Base class for every error this gem raises | +| `SnapDiff::Error` | Base class for every error this gem *defines* — see the caveat below. Misuse still surfaces as plain Ruby: bad arguments raise `ArgumentError`, and a missing image backend raises `RuntimeError` | | `SnapDiff::ExpectationNotMet` | A screenshot did not match its baseline | | `SnapDiff::UnstableImage` | No stable capture within `stability_time_limit` / `wait` | | `SnapDiff::WindowSizeMismatchError` | Browser window is not the configured `window_size` | @@ -176,6 +188,17 @@ integration require; a few objects need their own require, noted below. | `SnapDiff::Capture::Viewport` | Per-capture viewport check seam (`require "snap_diff/capture/viewport"`) | | `SnapDiff.serve` | Point Capybara at a static site directory (`require "snap_diff/static"`) | +> **`rescue SnapDiff::Error` does not catch a failed assertion under the framework +> integrations.** `SnapDiff::ExpectationNotMet` — the row you most want to rescue — is +> converted before it reaches your code: the Minitest integration re-raises it as +> `Minitest::Assertion` ([`integrations/minitest.rb`](../lib/snap_diff/integrations/minitest.rb)), +> and the RSpec integration as `RSpec::Expectations::ExpectationNotMetError` +> ([`integrations/rspec.rb`](../lib/snap_diff/integrations/rspec.rb)). That conversion is the +> point — it is what makes a mismatch a test failure rather than an error. You see a raw +> `SnapDiff::ExpectationNotMet` only when you drive `SnapDiff::DSL` yourself, with a bare +> `include SnapDiff::DSL` and no integration. `UnstableImage` and `WindowSizeMismatchError` +> are not converted and do arrive as `SnapDiff::Error`. + ## Compare two images without a browser Works on anything on disk — rendered PDFs, generated charts, CI artifacts: @@ -270,7 +293,7 @@ SnapDiff::Reporting.finalize! > by name. In 2.0 all of it still works and warns once per process (silence with > `SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`). Nothing > here migrates to a 2.1 shape — there is no 2.1 shape. If you maintain a driver, say so on -> [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166) before 2.1 ships. +> [the issue tracker](https://github.com/snap-diff/snap_diff-capybara/issues) before 2.1 ships. A driver is a plain object that does the image work. Include `SnapDiff::Driver` for the shared defaults, then implement the operations the comparison engine calls: diff --git a/test/unit/gemspec_packaging_test.rb b/test/unit/gemspec_packaging_test.rb new file mode 100644 index 00000000..e8bbee26 --- /dev/null +++ b/test/unit/gemspec_packaging_test.rb @@ -0,0 +1,47 @@ +# frozen_string_literal: true + +require "test_helper" + +# The gemspec's `spec.files` allow-list is the one part of the release that no +# other test looks at, and it has shipped broken twice: a missing +# Bundler.require entry file makes `gem "..."` load NOTHING and fail later with +# a confusing NameError, far from the cause. These assertions are about the +# PACKAGED file list, not about the working tree. +class GemspecPackagingTest < ActiveSupport::TestCase + GEMSPEC_PATH = File.expand_path("../../capybara-screenshot-diff.gemspec", __dir__) + + def spec + @spec ||= Gem::Specification.load(GEMSPEC_PATH) + end + + # Bundler requires the gem's OWN NAME. Each published name needs a file that + # matches it, or `Bundler.require` silently no-ops for that name. + test "ships a Bundler.require entry file for both published gem names" do + assert_includes spec.files, "lib/capybara-screenshot-diff.rb" + assert_includes spec.files, "lib/snap_diff-capybara.rb" + end + + test "ships the files rubygems.org and a consumer read" do + ["README.md", "LICENSE.txt", "CHANGELOG.md", "docs/UPGRADING.md", "docs/snapdiff.md"] + .each { |file| assert_includes spec.files, file } + end + + # Contributor-only docs describe releasing this gem and bin/dtest, neither of + # which a consumer has. + test "ships no contributor-only docs" do + refute_includes spec.files, "docs/RELEASE_PREP.md" + refute_includes spec.files, "docs/docker-testing.md" + end + + test "ships no build or development files" do + cruft = spec.files.grep_v(%r{\A(lib/|docs/|README\.md\z|LICENSE\.txt\z|CHANGELOG\.md\z)}) + assert_empty cruft + end + + # Every driver is an optional require. Adding a runtime dependency here is a + # decision, not an accident -- 2.1 makes ruby-vips one deliberately. + test "declares capybara as its only runtime dependency" do + runtime = spec.dependencies.select { |dependency| dependency.type == :runtime } + assert_equal ["capybara"], runtime.map(&:name) + end +end