[Python-Dev] UTF-8 Mode now also enabled by the POSIX locale
vstinner at redhat.com
Tue Aug 28 08:59:09 EDT 2018
While working on test_utf8_mode on AIX (bpo-34347) and HP-UX
(bpo-34403), I noticed that FreeBSD doesn't work properly with the
POSIX locale (bpo-34527). I also noticed that my implementation of my
PEP 540 "UTF-8 Mode" doesn't respect the PEP: the UTF-8 Mode should be
enabled by the POSIX locale, not only by the C locale.
I just modified Python 3.7 and master (future 3.8) to enable UTF-8
Mode if the LC_CTYPE locale is "POSIX":
I also fixed FreeBSD to support the "POSIX" locale as well (3.6, 3.7
and master branches).
Note: The C locale coercion (PEP 538) is only enabled if the LC_CTYPE
locale is "C".
To finish my list of recent Unicode issues, I'm also working on
bpo-34523: select the Python filesystem encoding before
Py_Initialize(). This change will allow embedders to force an encoding
instead of letting Python choose one for them.
More information about the Python-Dev