[Python-Dev] marshal (was:Buffer interface in abstract.c? )

[Greg Stein]

M.-A. Lemburg wrote:
> 
> Fred L. Drake, Jr. wrote:
> >
> > M.-A. Lemburg writes:
> >  > Aside: Is the buffer interface reachable in any way from within
> >  > Python ? Why isn't the interface exposed via __XXX__ methods
> >  > on normal Python instances (could be implemented by returning a
> >  > buffer object) ?
> >
> >   Would it even make sense?  I though a large part of the intent was
> > to for performance, avoiding memory copies.  Perhaps there should be
> > an .__as_buffer__() which returned an object that supports the C
> > buffer interface.  I'm not sure how useful it would be; perhaps for
> > classes that represent image data?  They could return a buffer object
> > created from a string/array/NumPy array.

There is no way to do this. The buffer interface only returns pointers
to memory. There would be no place to return an intermediary object, nor
a way to retain the reference for it.

For example, your class instance quickly sets up a PyBufferObject with
the relevant data and returns that. The underlying C code must now hold
that reference *and* return a pointer to the calling code. Impossible.

Fredrik's open/close concept for buffer accesses would make this
possible, as long as clients are aware that any returned pointer is
valid only until the buffer_close call. The context argument he proposes
would hold the object reference.

Having class instances respond to the buffer interface is interesting,
but until more code attempts to *use* the interface, I'm not quite sure
of the utility...

>... 
> Hmm, how about adding a writeable binary object to the core ?
> This would be useful for the __getwritebbuffer__() API because
> currently, I think, only mmap'ed files are useable as write
> buffers -- no other in-memory type. Perhaps buffer objects
> could be used for this purpose too, e.g. by having them
> allocate the needed memory chunk in case you pass None as
> object.

Yes, this would be very good. I would recommend that you pass an
integer, however, rather than None. You need to tell it the size of the
buffer to allocate. Since buffer(5) has no meaning at the moment,
altering the semantics to include this form would not be a problem.

Cheers,
-g

--
Greg Stein, http://www.lyra.org/