Bug report
Importing an extension module fails with UnicodeEncodeError if its path contains characters unencodable in UTF-8, e.g. surrogate escapes of a file name undecodable in the filesystem encoding.
$ .../nd\udcffwt/python -c "import _json"
Traceback (most recent call last):
File "<string>", line 1, in <module>
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' in position 45: surrogates not allowed
As a result, CPython cannot be run in a directory whose name contains undecodable bytes, and cannot even be built there -- make fails in checksharedmods. On 3.13 and 3.14 every import of a shared extension module from such a path fails.
The cause is the key of the extensions cache: hashtable_key_from_2_strings() in Python/import.c encodes the module file name with _PyUnicode_AsUTF8NoNUL(). It was added in gh-105699.
The raw content of the strings (UCS1, UCS2 or UCS4) can be used for the key instead of the UTF-8 encoding.
Linked PRs
Bug report
Importing an extension module fails with
UnicodeEncodeErrorif its path contains characters unencodable in UTF-8, e.g. surrogate escapes of a file name undecodable in the filesystem encoding.As a result, CPython cannot be run in a directory whose name contains undecodable bytes, and cannot even be built there --
makefails inchecksharedmods. On 3.13 and 3.14 every import of a shared extension module from such a path fails.The cause is the key of the extensions cache:
hashtable_key_from_2_strings()inPython/import.cencodes the module file name with_PyUnicode_AsUTF8NoNUL(). It was added in gh-105699.The raw content of the strings (UCS1, UCS2 or UCS4) can be used for the key instead of the UTF-8 encoding.
Linked PRs