[Python-Dev] str object going in Py3K

[Bengt Richter]

On Wed, 15 Feb 2006 18:57:26 -0800, Guido van Rossum <guido at python.org> wrote:

>On 2/15/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Guido van Rossum wrote:
>> > So how about
>> > openbytes? This clearly links the resulting object with the bytes
>> > type, which is mutually reassuring.
>>
>> That looks quite nice.
>>
>> Another thought -- what is going to happen to os.open?
>> Will it change to return bytes, or will there be a new
>> os.openbytes?
>
>Hm, I hadn't thought about that yet. On Windows, os.open has the
>ability to set text or binary mode. But IMO it's better to make this
>always use binary mode.
>
>My expectation is that the Py3k standard I/O library will do all of
>its own conversions on top of binary files anyway -- if you missed it,
>I'd like to get rid of any ties to C's stdio.
>
Would the standard I/O module have low level utility stream-processing generators
to do things like linesep normalization in text or splitlines etc? I.e., primitives
that could be composed for unforseen usefulness, like unix pipeable stuff?

Maybe they could even be composable with '|' for unixy left->right piping, e.g., on windows

    for line in (os.open('somepath') | linechunker | decoder('latin-1')): ...

where os.open('path').__or__(linechunker) returns linechunker(os.open('path')),
which in turn has an __or__ to do similarly. Just had this bf, but ISTM it reads ok.
The equivalent nested generator expression with same assumed primitives would I guess be

    for line in decoder('latin-1')(linechunker(binaryfile('path'))): ...

which doesn't have the same natural left to right reading order to match processing order.

Regards,
Bengt Richter