Skip to content

Only require jax to build where jaxlib has a wheel - #193

Open
jdeast wants to merge 1 commit into
exoplanet-dev:mainfrom
jdeast:build-jax-optional-on-platforms-without-jaxlib
Open

Only require jax to build where jaxlib has a wheel#193
jdeast wants to merge 1 commit into
exoplanet-dev:mainfrom
jdeast:build-jax-optional-on-platforms-without-jaxlib

Conversation

@jdeast

@jdeast jdeast commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

The JAX extension is already optional in CMakeLists.txt -- it probes for jax.ffi.include_dir() and, when that fails, prints Skipping JAX extension and builds everything else. But pip resolves [build-system] requires in an isolated environment before CMake runs, so that graceful path is unreachable and the pinned jax==0.8.0 makes the extension mandatory in practice.

Where jaxlib publishes no wheel, that makes the sdist unbuildable. macOS x86_64 is the case that bites: jaxlib's last wheel there is 0.4.38 and jaxlib ships no sdist, so the build fails while installing build dependencies, before any C++ is compiled:

× installing build dependencies for celerite2 did not run successfully.
  Collecting jax==0.8.0
  ERROR: Could not find a version that satisfies the requirement jaxlib<=0.8.0,>=0.8.0
         (from versions: 0.4.18, ..., 0.4.36, 0.4.38)
  ERROR: No matching distribution found for jaxlib<=0.8.0,>=0.8.0

This is not fixable downstream either: PIP_CONSTRAINT only narrows a requirement, so it cannot relax an == pin.

Why a marker rather than just dropping jax

Dropping jax from requires would work for the sdist but would silently change your releases. wheels.yml gets jax only via [build-system] requires, so without it CMake would take the "Skipping JAX extension" branch during cibuildwheel and the published wheels would quietly stop containing celerite2/jax/xla_ops -- building, installing, and passing CI exactly as before.

The marker keeps jax present on every platform you build wheels for and drops it only where jaxlib cannot be installed at all.

Verification

Run on a fork with this change:

  • Wheels: all three cibuildwheel jobs green, and all 15 wheels (macOS arm64, manylinux x86_64, win_amd64; cp311-cp314) contain celerite2/jax/xla_ops -- checked by inspecting the artifacts, since a missing extension would not otherwise show up as a failure.

  • Python: all 5 sessions green (core, jax, pymc, pymc_jax, tests-pymc).

  • Marker evaluated directly against the wheel matrix:

    platform wheels built? jax required
    linux x86_64 yes True
    linux aarch64 yes True
    macOS arm64 yes True
    windows AMD64 yes True
    macOS x86_64 no False

Found while getting a downstream package installable on Intel macOS. Happy to adjust the marker if you would rather express it differently.

🤖 Generated with Claude Code

The JAX extension is already optional in CMakeLists: it probes for
jax.ffi.include_dir() and, when that fails, prints 'Skipping JAX extension'
and builds everything else. But pip resolves [build-system] requires in an
isolated environment before CMake runs, so that graceful path is
unreachable and the pinned jax==0.8.0 makes the extension mandatory in
practice.

Where jaxlib publishes no wheel this makes the sdist unbuildable. macOS
x86_64 is the case that bites: jaxlib's last wheel there is 0.4.38 and
jaxlib ships no sdist, so the build fails while installing build
dependencies, before any C++ is compiled.

Adding an environment marker keeps jax present everywhere wheels are built
(the cibuildwheel matrix is linux x86_64, macOS arm64 and win_amd64, all of
which have jaxlib wheels), so released wheels are unaffected and still
contain the extension. Simply dropping jax from requires would instead have
let CMake silently skip the extension during release builds, which is why
this uses a marker rather than a removal.
jdeast added a commit to jdeast/EXOZIPPy that referenced this pull request Aug 14, 2026
Moves the Intel-Mac caps out of the CI-only override script and into
pyproject.toml as environment markers, so `poetry install` and
`pip install exozippy` work on macOS x86_64. Everything here was measured on
macos-15-intel across seven CI runs; see .github/workflows/intel-mac.yml.

The pins:

  jax / jaxlib / numpyro / blackjax   dropped on darwin+x86_64
  numba                              >=0.62.1,<0.63 there (0.65.1 elsewhere)
  numpy                              <2.4 there, following numba 0.62.1

Dropping the jax family rather than pinning it to 0.4.38 is measured, not
assumed, and it is the counterintuitive part: installing the newest jax that
platform CAN hold is strictly WORSE than installing none. exoplanet-core
reaches jax.ffi (public only since jax 0.5.0) and raises AttributeError,
which escapes its own `except ImportError` guard, so `import exozippy` dies
in components/orbit/orbit.py. With jax absent that guard catches a real
ImportError and everything works. Upstream fix in flight:
exoplanet-dev/exoplanet-core#146.

numba is capped because PyTensor 3 makes it a hard dependency and its last
macOS x86_64 wheels are 0.62.1 / llvmlite 0.45.1; past that the sdist wants
a matching LLVM. PyTensor accepts numba>=0.58, so this stays inside its
range. numpy follows from numba 0.62.1's own numpy<2.4.

Verified: every dependency parses as PEP 508 and exactly ONE variant of each
split requirement is selected on linux x86_64/aarch64, macOS arm64, macOS
x86_64 and windows AMD64 -- a marker pair that both-selects or
neither-selects would not show up until someone installed. poetry.lock
regenerated; it now carries both numba entries with complementary markers.

.github/scripts/intel_mac_deps.py is DELETED, as its docstring promised.
With the markers committed, the workflow installs with a plain
`pip install ".[gui]"` -- which is the point, since that is now exactly what
a user gets. Keeping the generator would have left a second copy of the pin
set to drift out of sync.

Two gaps remain, both documented rather than papered over, and both with
upstream fixes open:

  * `gp:` is unavailable -- celerite2's PyMC backend imports jax at module
    scope (exoplanet-dev/celerite2#194). The 10 kernel-building tests skip.
  * numpyro/blackjax are unavailable. nuts, ptde and nutpie all work.

And one manual step: celerite2 0.3.3 must be pre-built with
--no-build-isolation and BUILD_JAX=OFF, because its [build-system] requires
names jax==0.8.0 and build isolation resolves that where our pins cannot
reach (exoplanet-dev/celerite2#193). MACOS_INTEL_INSTALL.md is the runbook,
linked from README's supported-platforms section, and says which upstream
release deletes that step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant