Python Build by johannes-spies · Pull Request #292 · metatensor/metatomic

johannes-spies · 2026-07-21T09:15:20Z

This PR adds the infrastructure needed to build the metatomic_core Python wheel. Essentially, it's a port of the same functionality in metatensor (setup.py, CMakeLists.txt, and package-core.sh). The only difference is that we import metatensor in cmake_ext's run method to get the cmake_prefix_path which is required to build the native library.

The PR also makes an important change to the CI setup: now, containers for testing are built with Rust, similar to what metatensor does.

📚 Download documentation for this pull-request

⚙️ Download Python wheels for this pull-request (you can install these with pip)

Luthaf

Looks good overall, I'm not convinced we should switch to uv in CI, and we should for sure do it in a separate PR anyway

johannes-spies · 2026-07-22T20:01:52Z

Thank you for the review! You're right, the switch to uv was not necessary and I will revert it. I just tried to make the CI as similar as possible to track down the failure in the windows tests.

Luthaf

The wheel (as created by python -m build python/metatomic_core) contains configuration files to find nlohmann::json, which I don't think we want to make available through the to external users here.

[...]
adding 'metatomic/share/cmake/nlohmann_json/nlohmann_jsonConfig.cmake'
adding 'metatomic/share/cmake/nlohmann_json/nlohmann_jsonConfigVersion.cmake'
adding 'metatomic/share/cmake/nlohmann_json/nlohmann_jsonTargets.cmake'
adding 'metatomic/share/pkgconfig/nlohmann_json.pc'
[...]

Ideally they should not even be installed by the main metatomic CMakeLists.txt (this is not specific to Python) when we fetch the code from github ourself.

Can you have a look?

johannes-spies · 2026-07-27T09:54:01Z

Yes, you're right. 27675fd sets JSON_Install to OFF.

Luthaf · 2026-07-27T12:05:59Z

We still need to install the headers, just not the cmake config / .pc files

johannes-spies · 2026-07-31T10:48:06Z

I added the headers in 8dc3972. Also, I tried making the build-wheels.yml pipelines as close as possible to metatensor's pipeline! I hope everything is now very similar, minus uv.

Luthaf · 2026-08-06T15:11:35Z

+if (NOT nlohmann_json_FOUND)
+    # nlohmann_json was fetched and vendored by us: install its headers (used
+    # by metatomic's own public headers), but remove the CMake package config
+    # and pkg-config files it installs, since we don't want to advertise a
+    # system-wide nlohmann_json package to external users.
+    install(CODE "
+        file(REMOVE_RECURSE \"\${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_DATADIR}/cmake/nlohmann_json\")
+        file(REMOVE \"\${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_DATADIR}/pkgconfig/nlohmann_json.pc\")
+    ")
+endif()


Sorry for the back and forth here, I did not realized we were using the cmake files from our own metatomic-config.cmake …

Maybe the best solution here would be to install these to a different directory, i.e. to CMAKE_INSTALL_PREFIX/lib/metatomic/external instead of CMAKE_INSTALL_PREFIX? This way we can find the headers with normal cmake configuration, and also hide them from most users by default.

@RMeli what do you think here?

Sorry, I totally missed this ping! Hiding the installation sounds like a good idea. Another alternative is to simply manually install the headers, given that the library is header-only (there is also a single-header option).

I tried to include it header-only in 25d8f18.

RMeli · 2026-09-03T13:56:53Z

+        )
+
+
+def get_rust_version():


I'd use tomllib instead of manual parsing (which seems a bit fragile).

Good idea, but tomllib was added in Python 3.11 and the minimum python version is set to 3.10 in this project. I am not sure if we want to bump it because of metatomic-core's setup.py. I don't mind bumping it, so should we? @Luthaf, do you have an opinion on this?

Python 3.10 is going to be EOL in 2 months, so likely around the same time we release this. So I'm not against bumping the minimal version to 3.11 already!

RMeli · 2026-09-03T14:10:30Z

+if (NOT nlohmann_json_FOUND)
+    # nlohmann_json was fetched and vendored by us: install its headers (used
+    # by metatomic's own public headers), but remove the CMake package config
+    # and pkg-config files it installs, since we don't want to advertise a
+    # system-wide nlohmann_json package to external users.
+    install(CODE "
+        file(REMOVE_RECURSE \"\${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_DATADIR}/cmake/nlohmann_json\")
+        file(REMOVE \"\${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_DATADIR}/pkgconfig/nlohmann_json.pc\")
+    ")
+endif()


Sorry, I totally missed this ping! Hiding the installation sounds like a good idea. Another alternative is to simply manually install the headers, given that the library is header-only (there is also a single-header option).

RMeli · 2026-09-03T14:17:50Z

+RUN git config --global --add safe.directory /code
+
+# Download rustup-init asn install
+RUN curl https://sh.rustup.rs -sSf | sh -s -- -y --profile minimal --default-toolchain 1.96


Worth making the default toolchain an ARG (with default value 1.96)?

RMeli · 2026-09-03T14:17:59Z

+RUN git config --global --add safe.directory /code
+
+# Download rustup-init asn install
+RUN curl https://sh.rustup.rs -sSf | sh -s -- -y --profile minimal --default-toolchain 1.96


Same as above.

Co-authored-by: Rocco Meli <r.meli@bluemail.ch>

johannes-spies changed the base branch from main to metatomic-core July 21, 2026 09:17

johannes-spies requested a review from Luthaf July 22, 2026 09:02

Luthaf reviewed Jul 22, 2026

View reviewed changes

johannes-spies requested a review from Luthaf July 22, 2026 23:09

Luthaf reviewed Jul 27, 2026

View reviewed changes

Comment thread .github/workflows/build-wheels.yml Outdated

Comment thread .github/workflows/build-wheels.yml Outdated

Comment thread .github/workflows/build-wheels.yml Outdated

Luthaf force-pushed the metatomic-core branch from 49c982c to f810c3d Compare July 28, 2026 09:16

johannes-spies force-pushed the metatomic-core-python-build branch 2 times, most recently from 765ae6c to 8dc3972 Compare July 28, 2026 16:49

johannes-spies requested a review from Luthaf July 31, 2026 10:48

Luthaf reviewed Aug 6, 2026

View reviewed changes

Luthaf force-pushed the metatomic-core branch 2 times, most recently from a13f4cc to 173345c Compare August 28, 2026 12:18

Luthaf force-pushed the metatomic-core-python-build branch from c1498bd to 034a829 Compare August 28, 2026 12:23

Luthaf force-pushed the metatomic-core branch 2 times, most recently from 0c4521a to 1637cfb Compare August 28, 2026 13:06

Build the library in metatomic-core Python package

7577f1b

Luthaf force-pushed the metatomic-core-python-build branch from 034a829 to 7577f1b Compare August 31, 2026 08:02

Luthaf force-pushed the metatomic-core branch from 1f4f620 to 1637cfb Compare September 2, 2026 12:01

RMeli reviewed Sep 3, 2026

View reviewed changes

johannes-spies and others added 7 commits September 3, 2026 17:14

Update python/metatomic_core/setup.py

7064a51

Co-authored-by: Rocco Meli <r.meli@bluemail.ch>

Update scripts/package-core.sh

700169f

Co-authored-by: Rocco Meli <r.meli@bluemail.ch>

Update python/scripts/rustc-manylinux_2_28_x86_64/Dockerfile

0bb4f95

Co-authored-by: Rocco Meli <r.meli@bluemail.ch>

Update python/scripts/rustc-manylinux_2_28_aarch64/Dockerfile

0aa1897

Co-authored-by: Rocco Meli <r.meli@bluemail.ch>

Include as header-only library without metadata

25d8f18

Bump Python version to 3.11 and use tomllib

522fdaa

Make Rust version ARG

69b2af1

		)


		def get_rust_version():

Uh oh!

Conversation

johannes-spies commented Jul 21, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Luthaf left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

johannes-spies commented Jul 22, 2026

Uh oh!

Luthaf left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

johannes-spies commented Jul 27, 2026

Uh oh!

Luthaf commented Jul 27, 2026

Uh oh!

johannes-spies commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johannes-spies Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

johannes-spies commented Jul 21, 2026 •

edited by github-actions Bot

Loading

johannes-spies commented Jul 31, 2026 •

edited

Loading

johannes-spies Sep 3, 2026 •

edited

Loading