[Python-ideas] PEP 540: Add a new UTF-8 mode

[Steve Dower]

Passing universal_newlines will use whatever locale.getdefaultencoding() returns (which at least on Windows is useless enough that I added the encoding and errors parameters in 3.6). So it sounds like it'll only actually do Unicode on Linux if enough of the planets have aligned, which is what Victor is trying to do, but you can't force the other process to use a particular encoding. universal_newlines may become a bad choice if the default encoding no longer matches what the environment says, and personally, I wouldn't lose much sleep over that.

(As an aside, when I was doing all the Unicode changes for Windows in 3.6, I eventually decided that changing locale.getdefaultencoding() was too big a breaking change to ever be a good idea. Perhaps that will be the same result here too, but I'm nowhere near familiar enough with the conventions at play to state that with any certainty.)

Cheers,
Steve

Top-posted from my Windows Phone

-----Original Message-----
From: "Barry Warsaw" <barry at python.org>
Sent: ‎1/‎6/‎2017 14:04
To: "python-ideas at python.org" <python-ideas at python.org>
Subject: Re: [Python-ideas] PEP 540: Add a new UTF-8 mode

On Jan 05, 2017, at 05:50 PM, Victor Stinner wrote:

>I guess that all users and most developers are more in the "UNIX mode"
>camp. *If* we want to change the default, I suggest to use the "UNIX
>mode" by default.

FWIW, it seems to be a general and widespread recommendation to always pass
universal_newlines=True to Popen and friends when you only want to deal with
unicode from subprocesses:

    If encoding or errors are specified, or universal_newlines is true, the
    file objects stdin, stdout and stderr will be opened in text mode using
    the encoding and errors specified in the call or the defaults for
    io.TextIOWrapper.

Cheers,
-Barry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170106/38013bc8/attachment.html>