[Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Antoine Pitrou]

On Tue, 28 Jun 2011 10:41:38 -0400
Terry Reedy <tjreedy at udel.edu> wrote:
> On 6/28/2011 10:06 AM, Antoine Pitrou wrote:
> > On Tue, 28 Jun 2011 15:43:05 +0200
> > Victor Stinner<victor.stinner at haypocalc.com>  wrote:
> >>   - ISO-8859-1 os some FreeBSD systems
> >>   - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
> >> Western Europe, cp952 in Japan, ...
> >>   - ASCII if the locale is manually set to an empty string or to "C", or
> >> if the environment is empty, or by default on some systems
> >>   - something different depending on the system and user configuration...
> >
> > Why would utf-8 be the right thing in these cases?
> 
> Because utf-8 is the only way to write out any Python 3 text.

Er, no, you also have utf-16, utf-32, utf-7 (and possibly others,
including home-baked encodings).

> By default, writing and reading an str object should work on all Python 
> installations.

But that's only half of the problem. If the text is supposed to be read
or processed by some other program, then writing it in some encoding
that the other program doesn't expect doesn't really help. That's why
we use the locale encoding: because it's a good guess as to what the
system (and its users) expects text to be encoded in.

Regards

Antoine.