[Python-Dev] Buffer protocol for io.BytesIO?

[Antoine Pitrou]

On Fri, 3 Sep 2010 09:32:22 -0700
Guido van Rossum <guido at python.org> wrote:
> >> > It could not be resized, but it could be modified (same as what happens
> >> > with bytearrays today). Actually, the buffer itself would be writable,
> >> > and allow modifying the BytesIO contents.
> >>
> >> You may need to be careful with reads and writes while the buffer is
> >> exposed (e.g. throwing an exception until the buffer is released
> >> again). Perhaps the buffer accessor should be a context manager so it
> >> is easy to enforce prompt release?
> >
> > That's an interesting idea. I was planning to return a memoryview
> > object (in order to hide the intermediate object, and make it really
> > minimal), so perhaps the context protocol should be enabled on
> > memoryviews?
> >
> > (__enter__ would be a no-op, and __exit__ would release the internal
> > buffer and render it invalid, a bit like closing a file)
> 
> So far I'm  -0 on this. I'm not sure if it solves a real problem, and
> I think that if we were to add a way to define the scope of a buffer
> operation using a with-statement, it should work for all objects that
> support the buffer protocol (which IIUC means more than just
> memoryview). I'd be more enthusiastic about a separate context manager
> to wrap any such object.

Most objects don't have a dedicated Python method to return a buffer,
because they implement the buffer API implicitly at the C level, using
stack-allocated Py_buffer structs. Therefore, there would be no point in
a context manager for these objects (the buffer is released timely by
the consuming function). It's only when creating a persistent
Python-level wrapper for the Py_buffer struct (that is, a memoryview)
that heap memory management issues could come into play.

You are right that it's probably not a very important issue. Mutating
an object's buffer from several threads isn't recommended anyway. As
for resource consumption, what matters is when the original object
(which owns the memory area) gets deallocated, and there's little
chance that creating an additional memoryview makes a significant
difference.

Regards

Antoine.