[Python-Dev] str object going in Py3K

[M.-A. Lemburg]

Walter Dörwald wrote:
> Guido van Rossum wrote:
> 
>> On 2/16/06, M.-A. Lemburg <mal at egenix.com> wrote:
>>> What will be the explicit way to open a file in bytes mode
>>> and in text mode (I for one would like to move away from
>>> open() completely as well) ?
>>>
>>> Will we have a single file type with two different modes
>>> or two different types ?
>>
>> I'm currently thinking of an I/O stack somewhat like Java's. At the
>> bottom there's a class that lets you do raw unbuffered reads and
>> writes (and seek/tell) on binary files using bytes arrays. We can
>> layer onto this buffering, text encoding/decoding, and more.  (Windows
>> CRLF<->LF conversion is also an encoding of sorts).
>>
>> Years ago I wrote a prototype; checkout sandbox/sio/.
> 
> However sio.DecodingInputFilter and sio.EncodingOutputFilter don't work
> for encodings that need state (e.g. when reading/writing UTF-16).
> Switching to stateful encoders/decoders isn't so easy, because the
> stateful codecs require a stream-API, which brings in a whole bunch of
> other functionality (readline() etc.), which we'd probably like to keep
> separate. I have a patch (http://bugs.python.org/1101097) that should
> fix this problem (at least for all codecs derived from
> codecs.StreamReader/codecs.StreamWriter). Additionally it would make
> stateful codecs more useful in the context for iterators/generators.
> 
> I'd like this patch to go into 2.5.

The patch as-is won't go into 2.5. It's simply the wrong approach:
StreamReaders and -Writers work on streams (hence the name). It
doesn't make sense adding functionality to side-step this behavior,
since it undermines the design.

Like I suggested in the patch discussion, such functionality could
be factored out of the implementations of StreamReaders/Writers
and put into new StatefulEncoder/Decoder classes, the objects of
which then get used by StreamReader/Writer.

In addition to that we could extend the codec registry to also
maintain slots for the stateful encoders and decoders, if needed.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 17 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::