[Python-Dev] Buffer protocol for io.BytesIO?

[Guido van Rossum]

On Fri, Sep 3, 2010 at 10:03 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 3 Sep 2010 09:32:22 -0700
> Guido van Rossum <guido at python.org> wrote:
>> >> > It could not be resized, but it could be modified (same as what happens
>> >> > with bytearrays today). Actually, the buffer itself would be writable,
>> >> > and allow modifying the BytesIO contents.
>> >>
>> >> You may need to be careful with reads and writes while the buffer is
>> >> exposed (e.g. throwing an exception until the buffer is released
>> >> again). Perhaps the buffer accessor should be a context manager so it
>> >> is easy to enforce prompt release?
>> >
>> > That's an interesting idea. I was planning to return a memoryview
>> > object (in order to hide the intermediate object, and make it really
>> > minimal), so perhaps the context protocol should be enabled on
>> > memoryviews?
>> >
>> > (__enter__ would be a no-op, and __exit__ would release the internal
>> > buffer and render it invalid, a bit like closing a file)
>>
>> So far I'm  -0 on this. I'm not sure if it solves a real problem, and
>> I think that if we were to add a way to define the scope of a buffer
>> operation using a with-statement, it should work for all objects that
>> support the buffer protocol (which IIUC means more than just
>> memoryview). I'd be more enthusiastic about a separate context manager
>> to wrap any such object.
>
> Most objects don't have a dedicated Python method to return a buffer,
> because they implement the buffer API implicitly at the C level, using
> stack-allocated Py_buffer structs. Therefore, there would be no point in
> a context manager for these objects (the buffer is released timely by
> the consuming function). It's only when creating a persistent
> Python-level wrapper for the Py_buffer struct (that is, a memoryview)
> that heap memory management issues could come into play.
>
> You are right that it's probably not a very important issue. Mutating
> an object's buffer from several threads isn't recommended anyway. As
> for resource consumption, what matters is when the original object
> (which owns the memory area) gets deallocated, and there's little
> chance that creating an additional memoryview makes a significant
> difference.

Thanks for the explanation. Still -0.

-- 
--Guido van Rossum (python.org/~guido)