test: cut the golden fixture's duplication and fold three VM tests into one (#312)

[defangdevs]

Summary

Two independent reductions for #312, one per commit.

1. tests/golden stores identical rendered bytes once.

I measured what actually drives the fixture diff rather than guessing. The
web golden config is the vm config plus the web overlay, so 12 of the 16
files vm rendered were byte-for-byte copies of web's — including the three
biggest payloads (supervisor 46 KB, session CLI 21 KB, codex remote control
12 KB) — and AGENTS.robot.md was a third copy of the seeded guide. So a
two-line edit to one generated script landed twice in tests/golden. That is
the multiplier behind "the changes in tests/golden is often way more than the
code being changed".

bin/golden-snapshot.py now deduplicates at render time: the first path in
sorted order owns the bytes, and every other path that renders them is one line
in tests/golden/DUPLICATES. Fixture goes from 67 files to 55.

Replayed over every commit since 2026-07-20 that touched the fixture, that is
28% less golden churn overall — 20-50% on anything touching a shared payload
or the guide, and 0% on web-only changes, which is correct because nothing was
duplicated there:

commit	golden lines before	after
fix(sessions): move the supervisor's observations out of sessions.json	701	412
fix(sessions): lock sessions.json across every read-modify-write	520	351
docs(agents): type slash commands into your own pane	552	340
fix(sessions): follow /clear's session-id rotation on respawn	185	94
fix(webhook): launch claude with --channels	82	41
feat(web): download a session's transcript from its row (web-only)	333	333

No coverage is given up: the check still diffs the whole rendered snapshot, so a
copy that stops matching drops out of DUPLICATES and reappears as its own file
in the same diff. The header in DUPLICATES says exactly that, so a
disappearing line does not read as a cleanup.

2. Three VM tests that were the same test become one.

web-fail2ban.nix, self-serve-domain.nix and download-files.nix had the
same node definition — same services.agent-box block, same 12-line
password-hash activation script, same curl-only client node — and differed only
in the Caddyfile they lib.mkForce-swapped in. That is one capability per file
rather than one test per capability, and it spent three guest boots on three
routing shapes of one vhost.

They are now tests/web-surface.nix: one guest, one client, three named
subtest() blocks, every assertion and every hard-won comment carried over.
366 lines become 261, and CI goes from 7 VM checks to 5 — one --max-jobs 3
wave less.

Merging also adds coverage. The fail2ban subtest runs last (it bans the
client), which puts it after the self-serve subtest's systemctl reload caddy.service — so its "correct password still works" check now proves the
{$WEB_PASSWORD_HASH_AGENT} placeholder survives a reload. Neither split test
could see that: fail2ban never reloaded, and the self-serve Caddyfile carried no
placeholder.

What I deliberately did not touch

tests/webhook.nix (1949 lines) and tests/sessions.nix (1682) are the files
with the one-stanza-per-issue growth pattern, and an earlier comment on #312
proposed regrouping them into subtest() blocks. I decided against it here:

• They are not padding. 552 and 505 of those lines are comments explaining why
  an assertion exists, and the rest is ~380 dense assertions each. Coverage is
  what makes them long, and I could not find assertions worth deleting.
• The payoff is navigability only, and the price is a ~3600-line diff — the very
  thing this issue is about.
• refactor(module): systemd template units, %i = user (#154 Phase 3) #295 (Phase 3) and feat(agents): add opencode support #315 both edit those two files right now. Reordering them
  underneath would be a merge conflict for no behavior gain.

Regrouping them into subtest() blocks is worth doing once #295 lands, as a
pure-move commit with no assertion changes, so the diff is reviewable as a move.

A further lever, if you want it (not in this PR)

The remaining golden churn is inherent: a payload's bytes are in the fixture, so
an intentional change to modules/src/* shows up once in the source and once in
the fixture. It can be cut to near zero by recording payloads as
sha256 + the normalized store references they contain instead of full bytes. A
hash is an equally strong byte-stability lock (a no-op refactor still shows zero
diff), and the reference list still catches wiring changes — but you lose the
readable "here is how the rendered config changed" diff that #154 Phase 0 was
built for, and #295 is the current consumer of exactly that. Your call; I did not
want to remove a review capability you designed in while a refactor is mid-flight.

Closes #312

Test plan

• nix build .#checks.aarch64-linux.golden-snapshot — passes with the
  deduplicated fixture (and still passed before, so the fixture is
  cross-arch stable either way)
• All aarch64-native checks green: assemble-module-escaping,
  download-route, golden-snapshot, module-generated-up-to-date,
  module-single-file, multi-user, webhook-route
• nix eval .#checks.x86_64-linux.web-surface.drvPath evaluates; the three
  removed check names no longer resolve; settings-page,
  memory-protection, sessions, webhook still evaluate
• check-testscript.sh tests/web-surface.nix — the driver's own ty and
  ruff --select F gates on the testScript pass
• caddy adapt accepts the merged Caddyfile both with and without a
  ~/sites snippet present (empty import glob is a warning, as before)
• full CI — the VM tests are x86-only, so web-surface gets its first real
  boot here

🤖 Generated with Claude Code

https://claude.ai/code/session_01AMbiL3ux2XbUTrGQqP15U1