Support free-threaded CPython builds - #7
Conversation
The extension was always built against the stable ABI (`Py_LIMITED_API` / `py_limited_api=True`). The limited API is not available on free-threaded CPython 3.13+ builds, because the limited API headers omit the atomic operations that thread-safe reference counting needs, so the module failed to build there. Only request the stable ABI (and the `abi3` wheel tag) when `Py_GIL_DISABLED` is unset. Switch the module to multi-phase initialisation so it can declare `Py_MOD_GIL_NOT_USED`; the binding only returns a pointer to the static language table and holds no mutable module state, so it does not need the GIL. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CPython defines this macro in pyconfig.h on POSIX free-threaded builds but not on Windows, where it must be passed to the compiler. Without it the guard around the Py_mod_gil slot in binding.c is false on Windows and the module ends up re-enabling the GIL at import. Hoist the free-threading check and the macro list to module level so the three places that depend on it cannot drift apart. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ac21705f-befb-4606-9c83-657e3e9ea0dc
espressolee
left a comment
There was a problem hiding this comment.
I reviewed exact head ac79bb41506044d1f5fe346769d001f4c8a21b06 and do not see a blocking issue in this PR.
The two-file change is internally consistent: normal CPython builds retain the cp39-abi3 wheel path, while a free-threaded build drops the limited API, retains the interpreter-specific cp314-cp314t tag, and exposes Py_mod_gil = Py_MOD_GIL_NOT_USED through multi-phase initialization. I also checked the generated parser/scanner surface: parser tables are const and external scanner state is allocated per parser, so I did not find shared mutable state contradicting the marker in the exercised binding.
I independently built this exact head on Windows 11 x64 with CPython 3.14.7t. The result was a cp314-cp314t-win_amd64 wheel with a .cp314t-win_amd64.pyd; import kept the GIL disabled, and a 12-thread stress run completed 2400/2400 parses with zero errors. The exact-head wheel SHA256 was 14821a1d09227378cae991cf4e78e967844ca36d4dfa6c8955241eb0b0f63828. The base control failed at the expected limited-API/free-threaded incompatibility.
The only gap I see is nonblocking: the repository itself does not yet carry a free-threaded CI regression that asserts the GIL state and wheel tag. My approval is scoped to this exact head and the tested Windows x64/CPython 3.14.7t surface.
Problem
The Python extension used single-phase initialization (
PyModule_Create) and declared nothing about the GIL. On a free-threaded interpreter, importing a module that does not declare GIL support re-enables the GIL at runtime and emits a warning, which defeats the point of running that build.Change
Follows the C API Extension Support for Free Threading how-to.
PyInit__bindingreturnsPyModuleDef_Init(&module)instead ofPyModule_Create(&module), withm_sizechanged from-1to0. The module holds no state, so there is nothing to migrate.Py_mod_gilslot.Py_MOD_GIL_NOT_USEDis declared throughm_slots, which is the how-to's recommended form for multi-phase modules.Py_mod_gilis not part of the limited API, soPy_LIMITED_APIandpy_limited_apiare set only when the interpreter is not free-threaded, and the wheel tag stayscp39/abi3in that case. A free-threaded build produces a version-specific wheel instead.Py_GIL_DISABLEDon Windows. The how-to notes that CPython defines this macro automatically on POSIX free-threaded builds but not on Windows, where it has to be passed to the compiler. Without it the#ifdef Py_GIL_DISABLEDguard inbinding.cis false, the slot is dropped, and a free-threaded Windows build silently produces a module that re-enables the GIL.setup.pynow passes it explicitly wheneversysconfig.get_config_var("Py_GIL_DISABLED")is set. On POSIX that repeats the existing definition with the same value, which is a legal redefinition.Validation
Verified on CPython 3.14.7 with the GIL enabled, which is the configuration this change must not disturb:
pip install .succeeds and still produces_binding.abi3.so, so the stable-ABI build is unchanged.tree_sitter_ruby.language()returns aPyCapsuleand thequeries/*.scmfiles are still packaged.Not verified: no free-threaded interpreter was available, so the free-threaded and Windows paths are derived from the CPython how-to rather than executed. What is directly checked here is that the conditionals leave the default build byte-for-byte equivalent in shape.
Reference
Reimplements tree-sitter#286, plus the Windows
Py_GIL_DISABLEDdefinition, which that PR does not include.