maintenance: CI flake guard, stale paths, two-tier acceptance tolerance (3.14.20, 3.14.21)

[jdeast]

Three independent maintenance changes from the 8.6.17 follow-up list.

1. A failed diagnostic upload must not fail the job (3.14.21)

PR #246 went red on probe (3.13) with Failed to CreateArtifact: Unable to make request: ENOTFOUND after 13m48s of the probe's own work had passed. A DNS blip reaching GitHub's artifact endpoint turned a green run red.

That last part is the cost, and why it is worth fixing rather than re-running: it teaches people to re-run reds instead of reading them, in a repo whose CI discipline is "red means stop". A flake amplifier is worse than a flake — and if: always() means these steps run precisely when something else has already gone wrong, i.e. exactly when the real failure most needs to be the thing you see.

Both uploads are diagnostics, and each file says so itself. continue-on-error: true on the two of them, so a job's verdict comes from the work it did. The alternative considered and rejected — keep the failure but move the aggregate gate ahead of the upload — keeps a lost artifact visible but leaves the shard red, which is the misleading half.

2. Retire the last references to paths that no longer exist

All of these passed: they are arbitrary Parameter labels, docstring examples, or synthetic configs the generic config layer does not validate. Worth fixing for one reason — a reader who greps lens.t_E finds nothing in the source and then has to work out whether the test is stale or the code is.

test_param_specificity.py gets a docstring fix rather than a rewrite, and the reason is worth recording: its TWO_SOURCE_SYSTEM is a pre-split config that passes only because ConfigManager does not validate a lens block — the same property that had made four test_config_provenance tests inert until the 8.6.17 sweep. Here it is harmless, because the check under test is component-agnostic. So the scenario is labelled historical, with a note that the LIVE case for that check is now detrend_coeffs.

Left alone deliberately: anything already framed as history. Still open: test_latex_macro_xref.py:228 asserts _latex_varname("lens.12.t_E") == "ezlensonetwotE"; pointing it at a real path changes the expected mangled string, so it needs a run rather than a guess.

3. Two-tier acceptance tolerance keyed on a platform fingerprint (3.14.20)

ONE tolerance (rtol 1e-6 / atol 1e-3), sized to the worst cross-platform difference, applied everywhere — including on the machine that recorded the fixtures, where no regression smaller than that can be detected at all. A genuine 1e-4 nat change on the reference machine passed.

Each fixture now records the platform it was measured on. The recording machine is held tight; anything else gets the physics tolerance. No per-platform fixtures, no calibration through CI, and a new platform gets the loose tier automatically instead of no coverage.

Measured, not assumed. PR #246's macOS job dumped every term: 786 of 925 bit-identical, largest absolute difference 2.2e-06 nats, signs mixed 74/65 (scatter, not a directional offset), macOS on accelerate against Linux's scipy-openblas. For scale, 0.1 sigma on a 1-D Gaussian is ~5e-3 nats.

The strict tier is 1e-14, not zero, and that came out of running the generator's --check before touching a fixture. 12 of 13 reproduce exactly; OGLE_0383LD's RV:mulensinstrument.model comes back 26832.036090685935 -> 26832.03609068592 (5.4e-16 relative, the same delta every run). So that recorded value is about an ulp stale against current code, and rtol=0 would have failed the reference platform against its own fixtures. 1e-14 is a few ulp here — still eight orders tighter than the cross-platform tier.

The fixtures were not regenerated. Regeneration rewrites every recorded value from current code, and these are a pre-split recording that cannot be reproduced — it would have silently adopted that OGLE_0383LD ulp instead of leaving it visible. One key added per file, every other key verified byte-for-byte: 65 insertions, 0 deletions.

The fingerprint is coarse — system, machine, BLAS name, not version — so a point release that changes no result cannot silently demote the reference machine to the loose tier, and one that does change a result is caught by the loose tier anyway.

test_both_tolerance_tiers_are_reachable exists because every fixture here carries this machine's fingerprint, so the strict tier is the only one the suite exercises locally: a bug making is_reference_platform answer True unconditionally would be invisible while holding macOS eight orders too tight. It asserts all three directions (own / foreign / unstamped) with a fabricated fingerprint. Same shape as the vacuity failures in docs/testing.md — the guarded path passes and the unguarded one is never entered.

Verification

Pre-push hook green (ruff, ruff format, full suite). Acceptance 29 passed on the strict tier with the delta dump unchanged; 200 passed across the files touched in (2).

🤖 Generated with Claude Code