[Python-Dev] Fw: Behavior of buffer()

[Scott Gilbert]

--- Raymond Hettinger <python@rcn.com> wrote:
> GvR thought you guys might have some ideas on this one for me.
> 
> If I don't get any replies, I may have to rely on my own instincts and
> judgment and no one knows what follies might ensue ;)
> 
> [...]

I think buffers have a weird duality that they don't really want.  In one
case, the buffer object acts as a low level way to inspect some other
object's PyBufferProcs.  I'll call this BufferInspector.  In the other
case, the buffer object just acts like an array of bytes.  I'll call this
ByteArray.

So for a BufferInspector, you'd want slices to return new "views" into the
same object, and repetition doesn't make any sense.  If you wanted to copy
the data out of the object you're mucking with, you'd be explicit about it
- either creating a new string, or a new ByteArray.

For a ByteArray, I think you'd want slices to have copy behaviour and
return a new ByteArray.  Repetition also makes perfect sense.

Of course this all gets screwy when the object being inspected by the
BufferInspector sense is created solely to provide a ByteArray.  I see this
as an ugly workaround for arraymodule.c not allowing one to supply a
pointer/destructor when creating arrays.

The fact that either of these pretend to be strings is really convenient,
but I don't think it has much to do with the weirdness.  The fact that
either of these returns strings for any operation is somewhat weird.  For
the ByteArray sense of the buffer object, it's analagous to a list
slice/repetition returning a tuple.


Since the array module already has a way to create a ByteArray (and a
ShortArray, and...), buffer objects don't really need to duplicate that
effort.  Except creating an array from your own "special memory" (mmap,
DMA, third party API), and backwards compatibility in general.  :-)



BTW: I chuckled when I saw you post this the first time.  This topic seems
to draw a lot of silence.

I know that I would suggest deprecating the PyBufferObject to just being a
BufferInspector, and taking what little extra functionality was in there
and stuffing it into arraymodule.c.  Another solution would be to factor
PyBufferObject into PyBufferInspector and a "bytes" object.  A few months
ago, I was tempted to submit a PEP saying as much, but I think that would
have quietly fallen to the floor.  Nobody seems to like this topic too
much...

If you do go in and make changes to bufferobject.c, I've already submitted
two patches (fallen quietly to the floor) that fix some other classic
"buffer problems".  You might want to look at them.  Or not :-)



Cheers,
    -Scott


__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com