[Python-Dev] Printing and __unicode__

[M.-A. Lemburg]

Guido van Rossum wrote:
>>>Though it acts somewhat as if its default encoding is "ascii".  This
>>>is somewhat inconsistent: you can write arbitrary Unicode strings, but
>>>the Unicode won't be converted to ASCII.  ASCII is converted to
>>>Unicode though.
>>
>>It is the only case of a "pure Unicode" stream in Python, where the
>>underlying "native" sequence is not one of bytes, but one of Unicode
>>characters.
>>
>>The real problem is that the "orientation" (wide or narrow strings) is
>>determined by the things written into the stream.
>>
>>It might be more reasonable to have StringIO.ByteIO and
>>StringIO.UnicodeIO constructors, which both accept an encoding=
>>argument, and will convert objects of the wrong "orientation"
>>using that encoding (defaulting to the system encoding).
> 
> 
> I'm not sure about those names, but I agree that the encoding should
> be forced when the StringIO instance is created.  Given that using
> Unicode with these is currently fragile at best, maybe we should say
> that unless you give an encoding argument, it's a byte stream and
> doesn't allow Unicode at all?  That would be consistent with cStringIO.

+1

The fact that StringIO works with Unicode (and then only in the
case where you *only* pass Unicode to it) is more an implementation
detail than a true feature. cStringIO doesn't have this implementation
detail, so porting from StringIO to the much faster cStringIO
doesn't work at all for Unicode.

I think that StringIO and cStringIO should be regarded as
binary streams without any encoding knowledge. It is easy
enough to wrap these into Unicode aware streams using the
codecs.StreamReaderWriter class as is done in codecs.open().

That API already adds the .encoding attribute to the stream
object, BTW.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/