warn when the model does not start where the user asked - #252
Open
jdeast wants to merge 1 commit into
Open
Conversation
THE CONTRACT: when a user sets a value, the model produces that value or
says why it cannot. The second half did not exist, and the failure was
SILENT -- examples/ob09020 pinned mulensevent.t_E = 76.9 and started at
74.48, and the only way to discover that was to compile the graph by
hand. Measured across the shipped examples: 19 build, 14 user values are
not reproduced.
ModelAuditor.check_user_starts() reads every user-set initval off the
COMPILED GRAPH -- not p.value (a draw from the prior) and not p.initval
(the ledger, which is half of what is under test), both of which look
right while the model starts elsewhere. inspect_start prints the misses
after the startup table, beside the existing flat/unused warnings.
Two reasons, separated by the ledger rather than guessed:
ledger == user, built != user -> the DERIVATION cannot preserve it.
ob09020's t_E: recorded at rank 100, never overwritten, and still
3.15% out because the seed path is heliocentric while the graph is
geocentric (symbolic_physics.py's SEEDING APPROXIMATION).
ledger != user -> OVERSPECIFIED, and _last_solved_by names the equation
that overwrote it.
Calling the first "overspecified" would send the user hunting for a
conflicting pin that does not exist, so the distinction is load-bearing.
WHAT THE SWEEP FOUND. All 14 misses are the first kind; nothing in any
shipped example is being trampled, which is direct evidence the
precedence system is not the problem here. The error tracks pi_rel
exactly as the frame diagnosis predicts: ob09020 (pi_rel = 1.21 mas, a
747 pc lens) is ~10x the bulge events at pi_rel ~ 0.1. Two finds that
were previously invisible: gj1214's transit.MIRILRS.jitter_variance
(+49.5%) and galactic_model's star.BulgeTarget.mass, which starts at
10**-0.3 because a mass pin does not reach logmass.
Degrees are wrapped: the prototype reported four examples as broken over
bigomega/alpha pairs exactly 360 apart, and a warning block with false
positives is one nobody reads. The ledger is converted through the
Parameter before comparison -- it stores INTERNAL units (config.py:890)
while the file is in user units, so a raw compare calls every converted
parameter overspecified (star.ra: 4.6095 rad against a written 264.105
deg). The first draft did exactly that; the regression test fails on it.
examples/ob09020 goes back to PUBLISHED VALUES with the justifying prose
removed -- a params file is a template, and a reader cannot tell a
transcribed number from a fitted one when every entry carries a defense.
pi_E_N returns to Skowron's -0.022 from a hand construction, q to the
printed 0.273, and theta_E = 2.95 is now pinned. The set does not close,
which is the normal case for marginal medians from one table plus
physical parameters from another paper, and the new report says so
instead of the file hiding it behind tuned numbers.
bigomega is the one entry that cannot be "the published value": Yee's
Omega_node = -7.767 is in their frame, and the documented mapping
consumes phi_pi, which is a model output. Re-converting against the
delivered phi_pi (162.068, from 158.795) gives 340.755, and the rule
reproduces the old 337.484 to 0.002 deg. Start logp +19,024 -> +36,397,
with alpha_0 = 189.082 (published 189.08) and s_0 = 0.4264 (0.4294)
unchanged. Measured: this fit cannot walk in bigomega at all -- +/-10
deg costs >1e6 nats and 60 deg costs 7e5, with the midpoint WORSE than
either end, so it is a dead mode and not a slow one.
The ob09020 acceptance fixture is regenerated for the deliberate seed
change (81 terms); the other 12 stay bit-identical.
Verified: 8 new unit tests (the units one fails on the pre-fix code);
85 passed across the acceptance, keplerian, polish, hot-chains and
wrapup-guard files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
jdeast
force-pushed
the
worktree-user-start-check
branch
from
September 11, 2026 05:22
ef7df4e to
11c4430
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a user sets a value, the model should produce that value or say why it can't. The second half did not exist, and the failure was silent:
examples/ob09020pinnedmulensevent.t_E = 76.9and started at 74.48, and the only way to find that out was to compile the graph by hand.Measured across the shipped examples: 19 build, 14 user values are not reproduced.
The check
ModelAuditor.check_user_starts()reads every user-setinitvaloff the compiled graph — notp.value(a draw from the prior) and notp.initval(the ledger, which is half of what is under test). Both look right while the model starts somewhere else.inspect_startprints the misses after the startup table, beside the existing flat/unused warnings.Two reasons, separated by the ledger rather than guessed:
_last_solved_bynames the equationCalling the first "overspecified" would send someone hunting for a conflicting pin that does not exist, so the distinction is load-bearing rather than cosmetic.
What the sweep found
All 14 misses are the first kind. Nothing in any shipped example is being trampled — every one is a value the engine faithfully kept and the derivation could not rebuild. That is direct evidence the precedence system is not the problem here, which is the opposite of where this investigation started.
The error tracks
pi_relexactly as the heliocentric/geocentric diagnosis predicts: ob09020 (pi_rel = 1.21 mas, a 747 pc lens) is ~10x the bulge events atpi_rel ~ 0.1. Two finds that were previously invisible:gj1214'stransit.MIRILRS.jitter_variance(+49.5%) andgalactic_model'sstar.BulgeTarget.mass, which starts at exactly10**-0.3because a mass pin never reacheslogmass.Two traps this walked into, both now regression-tested
Degrees are periodic. The prototype reported four examples as broken over
bigomega/alphapairs exactly 360 apart (352.57 vs -7.43, 210 vs -150, ...). Those are the same start. A warning block with false positives is one nobody reads.The ledger is in INTERNAL units (
config.py:890) while the params file is in user units, so comparing them raw calls every converted parameter overspecified —star.raholds 4.6095 rad against a written 264.105 deg. The first draft did exactly that.test_the_ledger_is_compared_in_user_unitsfails on the pre-fix code; that was verified by mutation, not assumed.examples/ob09020 back to published values
A params file is a template, and a reader cannot tell a transcribed number from a fitted one when every entry carries a paragraph defending it. The justifying prose is gone.
pi_E_Nreturns to Skowron's-0.022from a hand construction,qto the printed0.273, andtheta_E = 2.95is now pinned.The set does not close — marginal medians from one table plus physical parameters from another paper — and that is the normal case, not a fault. The new report says so instead of the file hiding it behind tuned numbers.
bigomegais the one entry that cannot be "the published value": Yee'sOmega_node = -7.767is in their frame, and the documented mapping consumesphi_pi, which is a model output. Re-converting against the deliveredphi_pi(162.068, from 158.795) gives340.755; the same rule reproduces the old337.484to 0.002 deg. Start logp +19,024 -> +36,397, withalpha_0 = 189.082(published 189.08) ands_0 = 0.4264(0.4294) unchanged — the published-only template starts better than the hand-tuned one did.Measured, because it decides whether this is seedable at all: this fit cannot walk in
bigomega.+/-10 degcosts >1e6 nats, 60 deg costs 7e5, and the midpoint is worse than either end — a dead mode, not a slow one, against PTDE's factor-200 flattening.Verification
8 new unit tests. 85 passed across the acceptance, keplerian, polish, hot-chains and wrapup-guard files. The ob09020 acceptance fixture is regenerated for the deliberate seed change (81 terms); the other 12 stay bit-identical. Pre-push hook green (ruff, ruff format, full suite).
Follow-up
The heliocentric/geocentric frame fix removes the largest remaining class of misses. It is now an improvement rather than a prerequisite, since the contract is satisfied by the warning.
🤖 Generated with Claude Code