[Python-Dev] Issue 3745 backwards incompatibility

[Michael Foord]

On 15/12/2009 11:23, M.-A. Lemburg wrote:
> Karen Tracey wrote:
>    
>> In testing some existing code with the 2.7 alpha release, I've run into:
>>
>>      TypeError: Unicode-objects must be encoded before hashing
>>
>> when the existing code tries to pass unicode objects to hashlib.sha1 and
>> hashlib.md5.  This is, I believe, due to changes made for issue 3745:
>>
>> http://bugs.python.org/issue3745
>>
>> The issue states the need to reject unencoded strings based on the fact that
>> one backend implementation (openssl) refused to accept them while another
>> (_sha256) assumed a utf-8 encoding.  The thing is, I cannot observe any such
>> difference using Python 2.5 or 2.6.  Instead of what is shown in the ticket
>> (which was done on a Python 3, I believe) I see, when I adjust the demo test
>> to use Python 2 syntax for "unencoded strings":
>>      
> I think this was a misunderstanding during the issue 3745 processing:
> the patch should not have been backported to trunk at all.
>
> For Python 3.x, the change was correct. For 2.x, a -3 warning
> would have been a better fit.
>
> Note that the non-OpenSSL SHA et al. modules have never defaulted to
> encoding to UTF-8 in Python 2.x. Python 2.x uses ASCII as default
> encoding. Only Python 3.x uses UTF-8 as default encoding.
>    

Doesn't Python 3 use the *platform* encoding as the default (which 
happens to be UTF-8 on sensible systems but is something truly horrible 
like CP1250 on Windows)? (So *assuming* a default encoding of UTF-8 is 
still incorrect on Python 3 if we are being consistent with other IO 
behaviour.)

All the best,

Michael

> I've added a note to the issue and reopened it.
>
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog