[Python-Dev] Issue #11022: locale.getpreferredencoding() must not set temporary LC_CTYPE

[Victor Stinner]

Hi,

I would like to know if it is too late (or not) to change the
behaviour of open() for text files (TextIOWrapper). Currently, it
calls locale.getpreferredencoding() to get the locale encoding by
default. It is convinient and I like this behaviour... except that it
changes temporary the LC_CTYPE locale to get the user prefered
encoding instead of using the current encoding. Python 3 does already
uses the user preferred encoding as the current encoding at startup.
Changing temporary the current encoding to the user preferred encoding
is useless and dangerous (may cause issues in multithreaded
applications). Setting the current locale using locale.setlocale()
does not affect TextIOWrapper, it's also surprising.

The change is just to replace locale.getpreferredencoding() by
locale.getpreferredencoding(False) in the io module.

Can I change this behaviour (before the first beta) in Python 3.3?

See the issue #11022 (and maybe also #6203) for the discussion on this change.

--

Leaving LC_CTYPE unchanged (use the "C" locale, which is ASCII in most
cases) at Python startup would be a major change in Python 3. I don't
want to do that. You would see a lot of mojibake in your GUIs and get
a lot of ugly surrogate characters in filenames because of the PEP
393.

Setting the LC_CTYPE to the user preferred encoding is just very
convinient and helps Python to speak to the user though the console,
to the filesystem, to pass arguments on a command line of a
subprocess, etc. For example, you cannot pass non-ASCII characters to
a subprocess, characters written by the user in your GUI, if your
current LC_CTYPE locale is C (ASCII): you get an Unicode encode error.

Victor