[Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

Georg Brandl g.brandl at gmx.net
Tue Jun 28 23:42:11 CEST 2011

On 28.06.2011 14:24, Terry Reedy wrote:

>> As discussed before on this list, I propose to set the default encoding
>> of open() to UTF-8 in Python 3.3, and add a warning in Python 3.2 if
>> open() is called without an explicit encoding and if the locale encoding
>> is not UTF-8. Using the warning, you will quickly notice the potential
>> problem (using Python 3.2.2 and -Werror) on Windows or by using a
>> different locale encoding (.e.g using LANG="C").


>> Should I convert this email into a PEP, or is it not required?
> I think a PEP is needed.

Absolutely.  And I hope the hypothetical PEP would be rejected in this form.

We need to stop making incompatible changes to Python 3.  We had the chance
and took it to break all kinds of stuff, some of it gratuitous, with 3.0 and
even 3.1.  Now the users need a period of compatibility and stability (just
like the language moratorium provided for one aspect of Python).

Think about porting: Python 3 uptake is not ahead of time (I don't want to
say it's too slow, but it's certainly not too fast.)  For the sake of porters'
sanity, 3.x should not be a moving target.  New features are not so much of
a problem, but incompatibilities like this one certainly are.

At the very least, a change like this needs a transitional strategy, like
it has been used during the 2.x series:

* In 3.3, accept "locale" as the encoding parameter, meaning the locale encoding
* In 3.4, warn if encoding isn't given and the locale encoding isn't UTF-8
* In 3.5, change default encoding to UTF-8

It might be just enough to stress in the documentation that usage of the
encoding parameter is recommended for cross-platform consistency.


More information about the Python-Dev mailing list