GMM sampler: dim-group keys were in the wrong frame, and a dimension could be in no group at all

[oshaughnessy-junior]

--sampler-method GMM returns a catastrophically wrong evidence. On a zero-likelihood fixture
whose exact answer is 0, rift_O4d at 94f352a returns ln Z = −28.7, −30.0, −30.0, −30.1,
−136.2 across five seeds. It finds the peak correctly, and reports sigma = 0.178 against that.

It is not a p_s normalization error. A gmm_dict key is a tuple of dimension indices, and
mcsamplerEnsemble numbers its dimensions differently depending on how it is driven:

driven by	dim i is
integrate(func, *args) — standalone	args[i]
setup() + draw_simplified(n, *params_ordered) — portfolio member	params_ordered[i]

The portfolio never calls member.integrate(). Both ILE drivers built one key set, from
sampler.params_ordered, and handed it to whichever consumer was configured — right for a
portfolio member, wrong for a standalone GMM. Two separate consequences follow.

Uncovered dimension — wrong evidence. The three named groups cover six dimensions. A run
with a seventh parameter (t_ref, i.e. no --time-marginalization) leaves one out.
MonteCarloEnsemble._sample() allocates sample_array with xpy.empty and writes only the
columns a dim-group names, so the uncovered column carried uninitialized memory into the
integrand and into calc_pdf, with no matching factor in sampling_prior_array. The value is
whatever the allocator returned, which is why it is a lottery rather than a constant. On base the
uncovered parameter is distance: dimension 6 is the one left out, and _sample always reads
indices in the args frame, where args[6] is distance.

Wrong parameters adapted — lost efficiency. Where the cover happens to be complete the
estimator stays unbiased and each group simply adapts the wrong parameters.

Measured

Zero-likelihood H1 fixture, exact ln Z = 0, base 94f352a.

lane	base	this branch
standalone GMM (5 seeds)	−28.7 … −136.2	−0.011
standalone GMM --time-marginalization	−0.017	−0.011
GMM --internal-gmm-correlate-all	+0.011	+0.011
portfolio AV,GMM (4 seeds)	−0.088 … +0.576	−0.031 … +0.022
AV --time-marginalization	−0.002	−0.002

--internal-gmm-correlate-all is a single full-dimension group, so the frame cannot matter and
nothing moves. The efficiency half, measured where both frames cover every dimension (6-dim
--time-marginalization case, narrow tilted sky ridge, 5 seeds, same sample budget):

frame	n_eff (min/median)	ln Z across-seed spread
params_ordered (base)	77 / 133	0.087 nats
by consumer (fixed)	214 / 241	0.015 nats

Both agree on ln Z, as they must.

Changes

• Both drivers pick the frame by consumer (mcsamplerEnsemble.dim_group_frame) and fill any
  leftover dimension (complete_dim_group_cover).
• integrate_likelihood_extrinsic_batchmode_lisa had the same defect plus a third failure
  mode: under --distance-marginalization its keys ran off the end of a shorter parameter
  list. Fixed the same way.
• The standalone driver validates the mapping at startup. analyze_event is wrapped in a bare
  except Exception that writes an empty .dat and exits 0 under
  --internal-make-empty-file-on-error, so a refusal raised from sampler.integrate() would
  have looked like a job that merely found nothing.
• MonteCarloEnsemble refuses a gmm_dict that does not partition range(d). It runs after
  validate_gmm_dict, which reports the more specific complaint and which callers exercise with
  a deliberately partial gmm_dict.
• mcsamplerEnsemble.integrate() named dim-group parameters from params_ordered while its keys
  index args, so validate_gmm_dict's messages named the wrong parameter.

Test

test/integrators/test_ensemble_dim_group_cover.py (14 tests), registered in
.travis/test-integrate.sh. Analytic targets (E_prior[exp(A cos phi_orb)] = I0(A)) for the
cover defect; the frame defect is pinned separately, because at 0.006–0.02 nats it sits an order
of magnitude below the evidence tolerance and no analytic assertion can see it.

Seven mutations in an isolated worktree, control clean each time: remove the guard; make
dim_group_frame ignore its driver; drop each of the three guard call sites individually; make
complete_dim_group_cover forget n_comp/gmm_adapt; remove the empty-key check; make
integrate() number dimensions by params_ordered.

Gates (ldas-grid, numpy backend, cupy does not import)

test-core-units.sh 580 passed / 13 skipped / 0 failed · test-lisa.sh 275 passed ·
test-posterior.sh rc=0 · test-integrate.sh 1 failed —
test_peak_local_runs_on_the_gpu_backend_and_matches_numpy, which fails identically on base
94f352a on this host: pytest.importorskip('cupy') does not skip under pytest 9.1.1 when cupy
raises a chained ImportError for a missing libcuda.so.1. Not reachable from this diff.

CIP is unaffected

It builds keys from low_level_coord_names and integrates with *low_level_coord_names — the
same frame — and its default grouping covers every dimension. On identical ILE input, CIP GMM
2.06572 vs CIP AV 2.06558.

Behaviour change to note: --internal-correlate-parameters accepts overlapping blocks
(e.g. 'mc,delta_mc mc,xi'), which the new guard now refuses. That configuration was
double-counting a dimension before, so the refusal is correct, but it turns a previously-running
job into a hard failure.

Not fixed, found in passing

• --sampler-method AV without --time-marginalization dies on a missing
  mcsamplerAdaptiveVolume.uniform_samp_vector.
• .travis/test-integrate.sh and test-posterior.sh write cdf*.png, integral_result*.dat,
  output-ILE-samples* and friends into the repo root.

🤖 Generated with Claude Code