Add debug support for MSVC builds - full change PR - #1262
Open
vadikmironov wants to merge 3 commits into
Open
vadikmironov wants to merge 3 commits into
vadikmironov wants to merge 3 commits into
Conversation
cpython-unix/build.py offers `debug` alongside noopt and pgo, and every Unix target declares it in ci-targets.yaml. Windows accepts only noopt/pgo and hardcodes configuration="Release", so there is no way to produce a debug interpreter. That matters for embedding. MSVC's pyconfig.h selects the import library from _DEBUG, which the debug CRT defines, so a consumer building with /MDd asks for python3XX_d.lib and cannot link against a release distribution. Follows the Unix spelling: `debug` is a peer of noopt and pgo in the option set, so debug+pgo is simply not a valid combination. The set is renamed to `options` to match, since debug is not an optimization. The rest is naming. A Debug configuration suffixes its artifacts with $(PyDebugExt), so PC/layout is passed --debug, and the executables, extension libraries and dependency libraries pick up the _d suffix. The tail-calling interpreter is disabled for Debug. PBS turns it on for 3.15 on x64, but [[msvc::musttail]] requires /O2 and under /Od MSVC reports C4737 for each dispatch site. CPython's early check for this was reverted and the build still fails as of September 2026. https://learn.microsoft.com/en-us/cpp/cpp/attributes#msvcmusttail python/cpython#148047 building.rst documents the option and the debug CRT requirement: the binaries import ucrtbased.dll and vcruntime*d.dll, which ship with Visual Studio and are not redistributable.
validate-distribution rejects a Windows `debug` build on two counts.
The PE allow list carries only release names:
error: python/install/python_d.exe loads illegal library python314_d.dll
error: python/install/python_d.exe loads illegal library VCRUNTIME140D.dll
error: python/install/python_d.exe loads illegal library ucrtbased.dll
error: python/install/DLLs/_sqlite3_d.pyd loads illegal library sqlite3_d.dll
The other two formats already accommodate this. The Mach-O list pairs
every release name with its debug counterpart (@rpath/libpython3.14.dylib
next to @rpath/libpython3.14d.dylib, and td for free-threaded debug), and
the ELF list pushes libpython{ver}d.so.1.0 and libpython{ver}td.so.1.0
unconditionally. Neither gates on build options, so neither does this.
Windows spells the suffix _d, after $(PyDebugExt), which combines with
the free-threaded t as python314t_d.dll.
ucrtbased.dll sits in alphabetical position rather than beside a
counterpart because it has none: a release build reaches the UCRT
through the api-ms-win-crt-* forwarders, while a debug build imports it
directly.
The abiflags check is also POSIX-shaped:
error: abiflags does not contain 'd'
CPython deliberately keeps the lowercase `abiflags` empty on Windows,
because it is widely used to calculate paths there. The uppercase
`ABIFLAGS` carries the marker, but only from 3.14:
https://github.com/python/cpython/blob/v3.14.7/Lib/sysconfig/__init__.py#L407
So consult EXT_SUFFIX instead, which every supported version derives from
the importer and which a debug build suffixes with _d. That also drops an
unwrap() that would panic on a distribution missing the key.
The disttests trip on one more count: test_ssl_with_keylogfile. CPython's
own test suite skips keylog on Windows debug builds to avoid mixing the
debug and release CRT, and from 3.12 set_keylog_filename refuses the
call with NotImplementedError. On 3.11 the unguarded call crashed the
interpreter on arm64. So the disttest is skipped for every Windows
debug distribution, as CPython does.
https://github.com/python/cpython/blob/v3.11.16/Lib/test/test_ssl.py#L4827
https://github.com/python/cpython/blob/v3.12.14/Modules/_ssl/debughelpers.c#L168
python/cpython#131839
The debug option is only useful if something builds it. Unix targets already carry debug and freethreaded+debug beside their optimised options; Windows carries neither. Each Windows target has pgo and freethreaded+pgo today, so each gains the matching debug pair, leaving the three in step with each other as they already were.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is a full implementation of debug support for Windows build (addresses #26 in full) and combines three commits together only for the purposes of review. Should maintainers have a preference to proceed with it, happy to assist, but otherwise I will be raising a set of three separate PRs for each commit to be discussed and merged.
1. Add a debug build option for Windows (
cpython-windows/build.py,docs/building.rst)debugis added to the option set besidenooptandpgo, with the same ordering as Unix build already uses. Same as with Unix, this removes illegal combinations likedebug+pgo. The set is renamed fromoptimizationstooptionsalso inline with the Unix build.configuration="Debug"is passed to msbuild andartifact_configaccordingly.PC/layoutgets--debugand that drives the_dartifact suffixes._dsuffix$(PyDebugExt)producesdocs/building.rstupdated with the debug option and the debug CRT requirement2. Accept a Windows debug distribution in the validator and disttests (
src/validation.rs,pythonbuild/disttests/__init__.py)ucrtbased.dll,VCRUNTIME140D.dll,VCRUNTIME140_1D.dll), the_dinterpreter DLLs andsqlite3_d.dllabiflagscheck consultsEXT_SUFFIXon Windows (comments as to why are embedded inn the patch)test_ssl_with_keylogfileis skipped on Windows debug builds following CPython's suite skipping keylog there3. Build the Windows debug configurations in CI (
ci-targets.yaml)pgoandfreethreaded+pgo; each gainsdebugandfreethreaded+debugEvidence
Every configuration added in the third patch was built, validated and tested: 26 jobs, VS 2022 everywhere except x64 3.15 on VS 2026, arm64 on
windows-11-armthroughvcvarsamd64_arm64.bat(I see #1261 raised and I will try my testbed on it too as it may be relevant for my changes).debugfor 3.10 to 3.15 on x64, x86 and (from 3.11) arm64;freethreaded+debugfor 3.13 to 3.15 on all three. All green: https://github.com/vadikmironov/omniglot-bazel-starter/actions/runs/34095975591Each job ran exactly what
windows.ymlruns:validate-distributionand thentest-distribution.pythrough the distribution's own interpreter (22 tests, 12 skipped, OK on every job). The archives carrypython314_d.dllandlibs/python314_d.lib,python_d.exenext to the canonicalpython.execopy the existing code makes,_d.cp314-win_amd64.pydasEXT_SUFFIX, andPy_DEBUG=1in the config vars; the free-threaded variant adds thetas expected. An embedder compiled/MDdagainst each links with only a/LIBPATH, andsys.gettotalrefcount()works at runtime.Four gotchas that are commented in the patches
The tail-calling interpreter cannot be built in Debug. A guaranteed tail call needs optimisations which are removed by
/Od. Microsoft documents[[msvc::musttail]]as requiring/O2(https://learn.microsoft.com/en-us/cpp/cpp/attributes#msvcmusttail), and CPython's gh-148047 shows the same flood of errors forbuild.bat -c debug --tail-call-interp. It is off for Debug.Py_DEBUGandABIFLAGSreach Windows config vars only from 3.14. 3.13's_sysconfig.config_vars()suppliesEXT_SUFFIX,SOABIandPy_GIL_DISABLED; 3.14 addsPy_DEBUGand emulatesABIFLAGSwhile keeping lowercaseabiflagsempty on Windows (https://github.com/python/cpython/blob/v3.14.7/Lib/sysconfig/__init__.py#L407). Validator readsEXT_SUFFIXinstead.From 3.14 a free-threaded Windows install no longer declares itself. 3.14 ships a static
PC/pyconfig.hthat only normalises an externally suppliedPy_GIL_DISABLED, so an embedder that does not pass/DPy_GIL_DISABLED=1gets a#pragma comment(lib, ...)naming the non-free-threaded import library. No changes per se and it applies to existing free-threaded builds already, but I will investigate the right course of actions separately.SSLContext.keylog_filenameis unsupported on Windows debug builds. CPython's suite skips every keylog test there (https://github.com/python/cpython/blob/v3.11.16/Lib/test/test_ssl.py#L4827), and from 3.12 the call raisesNotImplementedErrorunder_DEBUG(https://github.com/python/cpython/blob/v3.12.14/Modules/_ssl/debughelpers.c#L168, from python/cpython#131839).test_ssl_with_keylogfileis skipped on every Windows debug build same CPython.Limitations
ucrtbased.dllandvcruntime140d.dllare not redistributable by Microsoft. It only runs where Visual Studio is installed and it's exactly the same for CPython's debug binarie.ci-targets.yamllists for that version_d-prefixed; CPython behaviourcrt_featuresstaysvcruntime:140, though the debug binaries link the non-redistributable runtime. This is another open point and happy to tweak it further.CI
Once the third commit is merged, it will add 26 jobs to every push to main, 15 to 26 minutes each on free runners. Again, happy to drop or amend it if that is too much.