[Python-Dev] Deprecate the buffer object?
Mark Hammond
mhammond at skippinet.com.au
Thu Oct 30 17:21:06 EST 2003
Greg:
> Neil Schemenauer:
>
> > Okay. Perhaps I am missing something but would fixing it be as
> > simple as adding another field to the tp_as_buffer struct?
> >
> > /* references returned by the buffer functins are valid while
> > * the object remains alive */
> > #define PyBuffer_FLAG_SAFE 1
>
> That's completely different from what I had in mind, which was:
>
> (1) Keep a reference to the base object in the buffer object, and
>
> (2) Use the buffer API to fetch a fresh pointer from the
> base object each time it's needed.
>
> Is there some reason that still wouldn't be safe enough?
That would work, be less intrusive, and allow all existing code to work
unchanged. My only concern is that it does not go anywhere towards fixing
the buffer interface itself.
To my mind, the buffer object is fairly useless and I never use it - so I
really don't care. However, I do have real world uses for the buffer
interface. The most compelling is for async IO in the Windows world - I
need to pass a buffer Windows will fill in the background, and the buffer
interface provides the solution - except for the flaws that also drip down
to the buffer object, and leaves us with this problem.
Thus, my preference is to fix the buffer object by fixing the interface as
much as possible.
Here is a sketch of a solution, incorporating both Neil and Greg's ideas:
* Type object gets a new flag - TP_HAS_BUFFER_INFO, corresponding to a new
'getbufferinfoproc' slot in the PyBufferProcs structure (note - a function
pointer, not static flags as Neil suggested)
* New function 'getbufferinfoproc' returns a bitmask - Py_BUFFER_FIXED is
one (and currently the only) flag that can be returned.
* New buffer functions PyObject_AsFixedCharBuffer, etc. These check the new
flag (and a type lacking TP_HAS_BUFFER_INFO is assumed to *not* be fixed)
* Buffer object keeps a reference to the existing object (as it does now).
Its getbufferinfoproc delegates to the underlying object.
* Buffer object *never* keeps a pointer to the buffer - only to the object.
Functions like tp_hash always re-fetch the buffer on demand. The buffer
returned by the buffer object is then guaranteed to be as reliable as the
underlying object. (This may be a semantic issue with hash(), but
conceptually seems fine. Potential solution here - add Py_BUFFER_READONLY
as a buffer flag, then hash() semantics could do the right thing)
After all that, I can't help noticing Greg's solution would be far less work
<wink>,
Mark.
More information about the Python-Dev
mailing list