[ python-Bugs-1115989 ] xmlrpclib: wrong decoding in '_stringify'

[SourceForge.net]

Bugs item #1115989, was opened at 2005-02-04 07:25
Message generated for change (Comment added) made by lemburg
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1115989&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Dieter Maurer (dmaurer)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: xmlrpclib: wrong decoding in '_stringify'

Initial Comment:
'_stringify' tries to convert a unicode string 
into a pure ascii string by 'str(string)'. 
 
This can lead to catastrophic behaviour when 
Python's "defaultencoding" is not "ascii". 
 
The attached patch uses "string.encode('ascii')" 
instead 
to become independent of the default encoding. 
 

----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2005-02-16 10:26

Message:
Logged In: YES 
user_id=38388

+1 on the comment "avoid, if possible; may break external
libraries; use encoded streams and explicit encodings instead".

The "may go away in the future" is not true and, as far as I
remember, was never taken into consideration (remember that
we even wanted the default encoding to be determined by the
locale at one point, but then quickly dropped that idea
again... there's still code in site.py that enables this).
Setting the default encoding to something other than ASCII
is only possible  in site.py and sitecustomize.py - for a
good reason. The site maintainer should be able to use a
different default if needed for whatever reason.

OTOH, libraries should not assume the ASCII setting of the
default encoding and always make their assumption explicit :-)


----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2005-02-16 08:32

Message:
Logged In: YES 
user_id=38376

Adding a brief "avoid, if possible; may break external libraries, 
may go away in the future; use encoded streams and explicit 
encodings instead" note to the "sys" documentation might be 
a good idea, though.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2005-02-16 08:22

Message:
Logged In: YES 
user_id=3066

Agreed.  It's use in the test was simply to duplicate the 
environment the xmlrpclib code failed in. 
 
Getting all the right people to agree that it should be 
completely removed and the default encoding always set to 
ASCII is a separate issue, though.  This bug was a 
consequence of the existing, documented flexibility.  As 
long as that exists, the standard library has to deal with 
it. 
 

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2005-02-16 08:08

Message:
Logged In: YES 
user_id=38376

(footnote: like any other global interpreter setting, changing 
the default encoding is a bad idea in general.   the 
setdefaultencoding hook was added for experimentation 
during unicode development, and the idea was to remove it 
when things had settled down.  too bad the documentation 
doesn't reflect this, though...).

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2005-02-11 19:06

Message:
Logged In: YES 
user_id=3066

Fixed for Python 2.4.1 and 2.5:

Lib/xmlrpclib.py  1.36.2.1, 1.40
Lib/test/test_xmlrpc.py  1.5.4.1, 1.7

I committed the attached patch and added a unit test. 
Python 2.3 got left out since this doesn't seem critical
enough to destabilize old applications.  I won't strongly
object if someone backports it to 2.3.6, if they actually
think there's going to be a 2.3.6.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1115989&group_id=5470