[Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?
mal at egenix.com
Wed Jun 29 10:18:59 CEST 2011
Victor Stinner wrote:
> Le mardi 28 juin 2011 à 16:02 +0200, M.-A. Lemburg a écrit :
>> How about a more radical change: have open() in Py3 default to
>> opening the file in binary mode, if no encoding is given (even
>> if the mode doesn't include 'b') ?
> I tried your suggested change: Python doesn't start.
No surprise there: it's an incompatible change, but one that undoes
a wart introduced in the Py3 transition. Guessing encodings should
be avoided whenever possible.
> sysconfig uses the implicit locale encoding to read sysconfig.cfg, the
> Makefile and pyconfig.h. I think that it is correct to use the locale
> encoding for Makefile and pyconfig.h, but maybe not for sysconfig.cfg.
> Python require more changes just to run "make". I was able to run "make"
> by using encoding='utf-8' in various functions (of distutils and
> setup.py). I didn't try the test suite, I expect too many failures.
This demonstrates that Python's stdlib is still not being explicit
about the encoding issues. I suppose that things just happen to work
because we mostly use ASCII files for configuration and setup.
> Then I tried my suggestion (use "utf-8" by default): Python starts
> correctly, I can build it (run "make") and... the full test suite pass
> without any change. (I'm testing on Linux, my locale encoding is UTF-8.)
I bet it would also with "ascii" in most cases. Which then just
means that the Python build process and test suite is not a good
test case for choosing a default encoding.
Linux is also a poor test candidate for this, since most user setups
will use UTF-8 as locale encoding. Windows, OTOH, uses all sorts of
code page encodings (usually not UTF-8), so you are likely to hit
the real problem cases a lot easier.
Professional Python Services directly from the Source (#1, Jun 29 2011)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
More information about the Python-Dev