Skip to content

portfolio: make sampling_density the member p_s contract, and give mcsamplerGPU one - #356

Open
oshaughnessy-junior wants to merge 2 commits into
rift_O4dfrom
claude/portfolio-ps-contract
Open

oshaughnessy-junior wants to merge 2 commits into
rift_O4dfrom
claude/portfolio-ps-contract

Conversation

@oshaughnessy-junior

@oshaughnessy-junior oshaughnessy-junior commented Sep 16, 2026

Copy link
Copy Markdown
Owner

Rebased onto rift_O4d now that #354 has merged. It landed the same three
_pdf_norm[p] = 1.0 hunks independently, so those are dropped here in favour of its
version. What remains is the portfolio contract, reset_sampling, and the tests.

mcsamplerPortfolio builds q_mix from each member's sampling_density().
mcsamplerGPU had none, so any portfolio holding one fell back to the stratified
per-member joint_p_s, which needs every member to report a normalized density.
mcsamplerAdaptiveVolume reports V_s/V, larger by V_s**2, and corrects for it in
its own integrate_log.

Decided: sampling_density() is the contract and must be a normalized density.
joint_p_s stays each sampler's private scale, so AV is unchanged. mcsamplerGPU
gains sampling_density(). reset_sampling now restores _pdf_norm with the pdf,
which #354's fix left stale. The stratified fallback refuses;
portfolio_allow_stratified_density=True restores it.

Production is unaffected

The question here is AV alone, across the normalization changes that just landed.
GW240426 CIP on all.net, 3 replicates at each of three commits:
b281ccff2 (before #354), 94f352ad8 (rift_O4d tip, #354 + #355), and this branch.
Noise floor from same-commit pairs. Every pairwise JS is inside it.

sampler JS noise floor pre354 -> tip tip -> branch pre354 -> branch
AV (GWTC production), mc 0.0030-0.0071 0.0048 0.0048 0.0042
AV, eta 0.0034-0.0055 0.0045 0.0052 0.0050
default adaptive_cartesian, mc 0.0034-0.0059 0.0058 0.0056 0.0051
adaptive_cartesian_gpu, mc 0.0043-0.0059 0.0054 0.0048 0.0043

AV ln Z: 28.2369 (sd 0.0098), 28.2314 (sd 0.0058), 28.2426 (sd 0.0118). Largest
difference 0.011, one replicate spread.

The defect, and the fix

Constant integrand, non-square box, prior integral 5, exact ln Z = 1.609438,
adaptation running, 16 seeds, same settings both sides:

portfolio base 94f352ad8 branch
[AV] 1.6094 (sd 0.0000) 1.6094 (sd 0.0000)
[GPU] 1.6134 (sd 0.1158) 1.6134 (sd 0.1158)
[GMM] 1.6207 (sd 0.0093) 1.6207 (sd 0.0093)
[GPU,GPU] 1.5891 (sd 0.0691) 1.5891 (sd 0.0691)
[AV,GMM] 1.6092 (sd 0.0026) 1.6092 (sd 0.0026)
[GPU,GMM] 1.6165 (sd 0.0093) 1.6108 (sd 0.0151)
[AV,GPU] 0.4458 (sd 0.0326) 1.6105 (sd 0.0027)
[GPU,AV] 0.4330 (sd 0.0231) 1.6090 (sd 0.0028)

Five rows are byte-identical. Only the two AV+GPU orderings were broken, by 1.17
nats. [GPU,GMM] is a wash: both members already reported a normalized joint_p_s,
so the stratified fallback was unbiased there. Forming q_mix moves its mean 0.006
toward exact and raises the seed spread from 0.009 to 0.015.

Verified

test/integrators/test_portfolio_member_density.py, 14 cases, in
.travis/test-integrate.sh with the collection-count guard #354 added. Eight
mutations in a separate worktree, one per part of the fix, all caught.

Risk: the refusal raises inside integrate_log, so a member whose sampling_density
returns None late in a run now aborts it instead of finishing with a biased
evidence. No shipped member does that. reset_sampling is called only from tests and
one demo, so that half is a latent fix.

Adjacent, not fixed here

mcsamplerNFlow.draw_simplified returns (rv, p_s, p_prior); every other member
returns (p_s, p_prior, rv). It cannot be a portfolio member today. No nflows here,
so it is unexercised.

🤖 Generated with Claude Code

@oshaughnessy-junior
oshaughnessy-junior deployed to private-review-dispatch-rift September 16, 2026 11:57 — with GitHub Actions Active
@oshaughnessy-junior
oshaughnessy-junior force-pushed the claude/portfolio-ps-contract branch from bd37fbb to e7d6ee9 Compare September 16, 2026 11:59
@oshaughnessy-junior
oshaughnessy-junior deployed to private-review-dispatch-rift September 16, 2026 11:59 — with GitHub Actions Active
…samplerGPU one

mcsamplerPortfolio builds its balance-heuristic mixture denominator q_mix from each
member's sampling_density(X).  mcsamplerGPU did not implement that method, so every
portfolio containing one fell back to the legacy STRATIFIED denominator, which uses
each sample's own draw_simplified() joint_p_s.  That is unbiased only when every
member's joint_p_s is a normalized density.  mcsamplerAdaptiveVolume's is not: it
reports V_s/V, larger by V_s**2 than the density it draws from, and compensates inside
its own integrate_log.  Pooling the two mixed weights on scales differing by V_s**2.

Constant integrand, non-square box, prior integral 5, exact ln Z = 1.609438,
adaptation running, 16 seeds, same settings on both sides:

    portfolio       base 94f352a        branch
    [AV]            1.6094 (sd 0.0000)    1.6094 (sd 0.0000)
    [GPU]           1.6134 (sd 0.1158)    1.6134 (sd 0.1158)
    [GMM]           1.6207 (sd 0.0093)    1.6207 (sd 0.0093)
    [GPU,GPU]       1.5891 (sd 0.0691)    1.5891 (sd 0.0691)
    [AV,GMM]        1.6092 (sd 0.0026)    1.6092 (sd 0.0026)
    [GPU,GMM]       1.6165 (sd 0.0093)    1.6108 (sd 0.0151)
    [AV,GPU]        0.4458 (sd 0.0326)    1.6105 (sd 0.0027)
    [GPU,AV]        0.4330 (sd 0.0231)    1.6090 (sd 0.0028)

Five rows are byte-identical.  Only the two AV+GPU orderings were broken, by 1.17
nats.  [GPU,GMM] is a wash: both members already reported a normalized joint_p_s, so
the stratified fallback was unbiased there.

DECIDED: sampling_density() is the contract and must return a properly normalized
density; draw_simplified()'s joint_p_s stays each sampler's private scale.  AV's
reported p_s and its evidence are NOT changed -- its sampling_density already returned
the true density, and moving its internal scale would move production results for
nothing.  This branch touches mcsamplerAdaptiveVolume with comments only.

reset_sampling now restores _pdf_norm alongside the pdf.  #354 set _pdf_norm to 1 when
the adapted proposal is installed, which is right, but reset_sampling puts the
unnormalized analytic pdf back without it, leaving the reported density too large by
that mass.  Only tests and one demo call reset_sampling, so this is a latent fix.

The stratified fallback is now a refusal by default (opt out with
portfolio_allow_stratified_density=True), so the next member without a
sampling_density cannot reintroduce the bias silently.  That also closes two gaps the
driver warning recorded: a portfolio in which EVERY member lacked the method did not
warn, and an unformed q_mix left credit_per_sample identically zero, killing the
draw-allocation signal under portfolio_adaptive_alloc with quality_signal='credit'.

Production is unaffected.  GW240426 CIP on all.net, 3 replicates at each of
b281ccf (pre-#354), 94f352a (tip) and this branch: every pairwise 1-D JS in mc and
eta is inside the same-commit noise floor, for --sampler-method AV, for the default
adaptive_cartesian, and for adaptive_cartesian_gpu.  AV ln Z 28.2369 / 28.2314 /
28.2426, largest difference 0.011 against replicate spreads of 0.006-0.012.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@oshaughnessy-junior
oshaughnessy-junior force-pushed the claude/portfolio-ps-contract branch from e7d6ee9 to e89a06a Compare September 16, 2026 14:35
@oshaughnessy-junior
oshaughnessy-junior changed the base branch from claude/gpu-sampler-pdf-norm to rift_O4d September 16, 2026 14:35
@oshaughnessy-junior
oshaughnessy-junior marked this pull request as ready for review September 16, 2026 14:35
@oshaughnessy-junior
oshaughnessy-junior deployed to private-review-dispatch-rift September 16, 2026 14:35 — with GitHub Actions Active
@oshaughnessy-junior
oshaughnessy-junior deployed to private-review-dispatch-rift September 16, 2026 19:51 — with GitHub Actions Active
@oshaughnessy-junior
oshaughnessy-junior force-pushed the claude/portfolio-ps-contract branch 2 times, most recently from 0db8d36 to bc15c0d Compare September 16, 2026 20:12
@oshaughnessy-junior
oshaughnessy-junior deployed to private-review-dispatch-rift September 16, 2026 20:12 — with GitHub Actions Active
@oshaughnessy-junior
oshaughnessy-junior force-pushed the claude/portfolio-ps-contract branch from bc15c0d to 679d390 Compare September 16, 2026 20:36
@oshaughnessy-junior
oshaughnessy-junior deployed to private-review-dispatch-rift September 16, 2026 20:36 — with GitHub Actions Active
…into all four drivers

Four rounds of adversarial review of the commit before this one.  Each round found the
previous round's fix wrong in a new way, so all four are recorded here.

ROUND 1.  The refusal added by the previous commit aborted portfolios that cannot suffer the
bug, its escape hatch was reachable from no driver, it replaced a cheap setup-time warning
with an expensive mid-run abort, and a comment claimed every shipped portfolio member
implements sampling_density -- mcsamplerNFlow ships and does not.

ROUND 2.  The response gated on getattr(m, 'joint_p_s_is_normalized_density', True), so an
UNDECLARED member was assumed safe.  Plugins from known_pipelines and new samplers are the
population least likely to carry the attribute.  Measured: an undeclared member on AV's scale
pooled with another density-less member gave ln Z = 1.619388 against an exact 2.302585,
SILENTLY, where round 1's code had refused.  Fixing round 1 had turned a hard stop into a
quiet wrong answer.

ROUND 3.  Inverting the default was right, but the CLI flag added with it went to the two CIP
drivers only.  RIFT has FOUR portfolio-capable drivers: integrate_likelihood_extrinsic_batchmode
and its LISA twin also build portfolios, and ILE is the only one accepting a discovered
RIFT.integrator_plugins pipeline as a member.  Every ILE job in a DAG would have died at setup,
told to pass a kwarg no ILE command line accepted.

ROUND 4.  Closing the portfolio_use_mixture_density=False path broke
test_portfolio_balance_heuristic.py, which uses that path deliberately to EXHIBIT the bias
q_mix removes, failing test_integrator_studies.py in .travis/test-core-units.sh.  Adding the
kwarg to the LISA setup() call broke test_lisa_sampler_plumbing.py, which pinned that call as
a verbatim substring.  Both were green on the previous head.  The stratified arm of the study
now opts out explicitly, and the LISA gate asserts what it is for (the assembled dict reaches
setup()) rather than the exact spelling.

WHAT THIS COMMIT DOES.  The contract is a DECLARATION: mcsamplerGPU and mcsamplerEnsemble
declare joint_p_s_is_normalized_density = True, mcsamplerAdaptiveVolume declares False, and
absent-or-not-True means unknown, which refuses.  The setup() gate and the run-time backstop
use that same predicate.  setup() REMEMBERS the opt-out, so a driver passes it once instead of
threading it through every integrate() call site, and states it afresh each time so consent
cannot become sticky.  --sampler-portfolio-allow-stratified-density is a real CLI flag on all
four drivers.  portfolio_use_mixture_density=False selects the stratified MECHANISM and is no
longer read as consent to a biased evidence.

Both refusal messages now separate "declared False" from "never declared" and name only the
members that triggered them.  Round 4 showed the previous text called mcsamplerAdaptiveVolume
undeclared when it declares False, and then advised declaring it True, which would restore the
bias.  A mutation that names every chunk member instead of the guilty one survived all 33
tests, because pytest.raises(match=...) is satisfied by a message that also names innocents;
there is now a test asserting the complement.

Tests 14 -> 37.  Four tests added in earlier rounds were themselves false: an undeclared-member
stand-in set the attribute to None, which the permissive default also rejects, so it passed
with the defect present; a backstop test opted out at both gates so the backstop never ran; and
two were duplicate `def test_` names, which Python silently rebinds, so the stale copy shadowed
the new one while the collected count did not move.  A test now fails if any name in the file is
duplicated.

Numerics unchanged through all four rounds: the 16-seed constant-integrand table is identical in
all eight rows.  Every round changed which configurations are refused and what they are told,
not any evidence.  Production CIP on real GW240426 data runs clean for --sampler-method AV, the
driver default, and a portfolio.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@oshaughnessy-junior
oshaughnessy-junior force-pushed the claude/portfolio-ps-contract branch from 679d390 to f851a27 Compare September 16, 2026 21:37
@oshaughnessy-junior
oshaughnessy-junior deployed to private-review-dispatch-rift September 16, 2026 21:37 — with GitHub Actions Active
@oshaughnessy-junior
oshaughnessy-junior force-pushed the claude/portfolio-ps-contract branch from f851a27 to d1d7c7e Compare September 16, 2026 22:10
@oshaughnessy-junior
oshaughnessy-junior deployed to private-review-dispatch-rift September 16, 2026 22:10 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant