Colour the failure message when the terminal supports it

[pftg]

Follow-up to #264, which made the failure message legible. Colour is the next increment and is deliberately not in 2.0 — the structure had to land first, and colour on top of the old unlabelled-paths format would have been decoration on something unreadable.

Current output (after #264)

Screenshot does not match for 'changed': the change spans 100 of 6400 px (1.56% of the 80x80 image)
  changed region: [20.0,15.0,30.0,25.0] (left,top,right,bottom edges)
  judged against: no tolerance thresholds configured (any difference fails)
  baseline:           screenshots/changed.base.png
  actual:             screenshots/changed.png
  baseline annotated: screenshots/changed.base.diff.png
  actual annotated:   screenshots/changed.diff.png
  heatmap:            screenshots/changed.heatmap.diff.png

Readable, but uniform. Everything has the same visual weight, so the eye has no entry point — and this block sits directly above ~19 absolute backtrace frames.

What colour would carry

Not decoration — three specific jobs:

• Separate the block from the backtrace. Half the original "reads as noise" complaint was that the artifact block was indistinguishable from stack frames. Relative paths helped (feat: make the failure message legible #264); dimming the labels would finish it.
• Rank the numbers. The percentage is the number a human judges first; the raw pixel count is supporting detail.
• Distinguish the artifact kinds. baseline vs actual is the one pairing a reader has to hold in their head, and colour does that faster than alignment.

Requirements

• Respect NO_COLOR — no-color.org is the de-facto standard; presence of the variable, whatever its value, disables colour
• Only when the stream is a TTY. $stderr.tty? — CI logs, files and pipes must stay plain. There is currently no tty?/isatty/NO_COLOR/escape-sequence handling anywhere in lib/ (grepped), so this is greenfield
• Honour TERM=dumb
• A config knob to force on/off, for users whose CI does render colour (GitHub Actions does) and who want it
• No new dependency. Bare ANSI codes are a few lines; rainbow/pastel/colorize are not worth a runtime dep for this
• The HTML reporter and the [snap_diff] N verified… summary line are separate surfaces — decide explicitly whether the summary gets colour too, rather than doing it by accident
• Guard tests both directions: colour present on a TTY, absent when piped, when NO_COLOR is set, and when TERM=dumb. The absent-cases are the ones that break CI logs, so they matter more than the present-case

Accessibility

Colour must be additive, never load-bearing. Every distinction it makes must already be readable in the plain text — which is true of the current format, and must stay true. Around 8% of men have a red-green colour vision deficiency, and red/green is the obvious pairing for a diff tool.

Related: research/competitive-landscape.md records that colour-blind accessibility of diff artefacts was not assessed for any tool surveyed, and that none was observed documenting a colour-blind-safe palette. That gap applies to our own .diff.png annotations (currently a red rectangle) as much as to terminal output — worth a separate issue if anyone wants to take it on.

Prior art worth checking before implementing

RSpec, Minitest and RuboCop all colour test output and all handle the TTY/NO_COLOR/--no-color matrix already. Read how they detect rather than re-deriving it — RSpec in particular has had years of bug reports about colour leaking into CI logs.

[pftg]

Unscheduled — this is a maintainer idea, and maintainer ideas need proof

Removed from the 2.1 milestone and labelled needs-evidence. Standing rule (Paul, 2026-08-24): an idea originating from us is a hypothesis, not a decision. It gets filed and researched; it does not get scheduled on the strength of the reasoning that produced it.

I filed this one on my own argument about what colour would carry — ranking the numbers, separating the block from the backtrace. That is reasoning about what users would want, which is the weakest form of evidence and the one that has failed here repeatedly:

• "no demand for chunky_png" — never tested, and false: bootstrap_form (1,643 stars) runs it by default with no libvips in CI
• a four-judge panel voted 4/4 to flip the missing-baseline default — reversed by community evidence showing we would have been a minority of one
• RECORD_SCREENSHOTS=1 — documented in four files and printed in our own error message, implemented nowhere, for years

What would promote this to scheduled

Any one of, roughly strongest first:

• A user asks for it — issue, discussion, or colour-forcing found in a real config
• Real-world evidence that our plain output is hard to scan — e.g. someone piping it through a colouriser
• Evidence from a comparable tool that colour changed something measurable, not just that they have it
• A measurement that the current format costs a reader real time

Note the bar moved under this issue's feet in a useful way: #264 landed and the message is now labelled, has units, and states the applied threshold. The honest test is whether colour beats that, not whether it beats the unlabelled JSON blob this issue was written against. That may well be a smaller gap than it looked an hour ago.

Staying open. If someone hits this and says the output is hard to read, that is the evidence, and it gets scheduled the same day.

[pftg]

needs-evidence — researched. Verdict: accept the goal, reject the scope as written.

The maintainer idea was "colour the failure message." Two things came out of the research, and the second one changes what should actually be built.

1. Colour in Ruby test output is established practice, with a settled contract

• RSpec colours automatically when the output is a TTY; --color / --no-color force it, and config.color_mode takes :automatic (default) / :on / :off.
• Minitest ships minitest/color and a --color flag.
• NO_COLOR is the cross-language convention: if the variable is present and non-empty, emit no ANSI — motivated by accessibility (low contrast is unreadable for some users and on poor displays), not preference. FORCE_COLOR is its counterpart for pipes.
• CI is not a blanket "no colour": GitHub Actions has no TTY but does render ANSI in logs, and exposes GITHUB_ACTIONS for detection.

So the contract is not "colour it": it is auto-detect TTY → honour NO_COLOR → allow explicit override, and anything less is a regression for someone.

2. The failure message is the wrong place to put it — this is the blocker

comparison.rb:152 is @error_message ||= reporter.generate. That string becomes a Minitest assertion message, carried inside an exception. The gem never prints it — the test framework does.

At construction time the gem cannot know where that string ends up: a terminal, a JUnit XML attribute, a GitHub annotation, a saved log, a JSON reporter. ANSI escapes baked into an exception message leak into all of them, and NO_COLOR cannot save a consumer that never sees the variable. That is the same class of mistake as printing a path not derived from live state (#260) — emitting a decision into a context we do not control.

What is actually buildable

The gem does own a small number of writes it makes itself — reporting.rb:203–223 ($stdout.puts, the summary line and its siblings) and the warn calls. Those have a real $stdout, so they can honestly ask tty?, honour NO_COLOR/FORCE_COLOR, and detect GITHUB_ACTIONS.

Revised scope:

• Colour only the gem's own $stdout/stderr writes — the summary line first: 55 verified green, 1 changed amber, 2 new (not verified) the loud one.
• Detection: TTY → NO_COLOR (present, non-empty ⇒ off) → FORCE_COLOR → GITHUB_ACTIONS.
• Leave the assertion message uncoloured, and say why in a comment so it is not "fixed" later.
• No new dependency — this is a handful of escape codes, not a reason to add rainbow or pastel.

Keeping needs-evidence off now: the practice is evidenced and the contract is known. Staying on 2.1, since #264 already did the load-bearing work and this is decoration on top of it.

Sources: no-color.org · force-color.org · RSpec 3.6 colour modes · rspec-core#2230 on --tty/--color semantics · minitest-color · ANSI in GitHub Actions logs