[Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Michael Foord]

On 28/06/2011 15:36, Terry Reedy wrote:
> On 6/28/2011 10:02 AM, M.-A. Lemburg wrote:
>
>> How about a more radical change: have open() in Py3 default to
>> opening the file in binary mode, if no encoding is given (even
>> if the mode doesn't include 'b') ?
>>
>> That'll make it compatible to the Py2 world again
>
> I disagree. I believe
> S = open('myfile.txt').read()
> now return a text string in both Py2 and Py3 and a subsequent
> 'abc' in S
> works in both.

Nope, it returns a bytestring in Python 2. Mistakenly treating 
bytestrings as text is one of the things we aimed to correct in the 
transition to Python 3.

Michael

>
> > and avoid all the encoding guessing.
>
>> Making such default encodings depend on the locale has already
>> failed to work when we first introduced a default encoding in
>> Py2, so I don't understand why we are repeating the same
>> mistake again in Py3 (only in a different area).
>
> I do not remember any proposed change during the Py3 design discussions.
>
>> Note that in Py2, Unix applications often leave out the 'b'
>> mode, since there's no difference between using it or not.
>
> I believe it makes a difference now as to whether one gets str or bytes.
>
>> Only on Windows, you'll see a difference.
>
> I believe the only difference now on Windows is the decoding used, not 
> the return type.
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html