[Python-ideas] PEP 540: Add a new UTF-8 mode

[Victor Stinner]

Ok, I modified my PEP: the POSIX locale now enables the UTF-8 mode.

2017-01-05 18:10 GMT+01:00 Victor Stinner <victor.stinner at gmail.com>:
> A common request is that "Python just works" without having to pass a
> command line option or set an environment variable. Maybe the default
> behaviour should be left unchanged, but the behaviour with the POSIX
> locale should change.

http://bugs.python.org/issue28180 asks to "change the default" to get
a Python which "just works" without any kind of configuration, in the
context of a Docker image (I don't any detail about the image yet).


> Maybe we can enable the UTF-8 mode (or "UNIX mode") of the PEP 540
> when the POSIX locale is used?

I read again other issues and I confirm that users are looking for a
Python 3 which behaves like Python 2: simply don't bother them with
encodings. I see the UTF-8 mode as an opportunity to answer to this
request.

Moreover, the most common cause of encoding issues is a program run
with no locale variable set and so using the POSIX locale.

So I modified my PEP 540: the POSIX locale now enables the UTF-8 mode.
I had to update the "Backward Compatibility" section since the PEP now
introduces a backward incompatible change (POSIX locale), but my bet
is that the new behaviour is the one expected by users and that it
cannot break applications.

I moved my initial proposition as an alternative.

I added a "Use Cases" section to explain in depth the "always work"
behaviour, which I called the "UNIX mode" in my previous email.

Latest version of the PEP:
https://github.com/python/peps/blob/master/pep-0540.txt

https://www.python.org/dev/peps/pep-0540/ will be updated shortly.

Victor