Greg:
Neil Schemenauer:
Okay. Perhaps I am missing something but would fixing it be as simple as adding another field to the tp_as_buffer struct?
/* references returned by the buffer functins are valid while * the object remains alive */ #define PyBuffer_FLAG_SAFE 1
That's completely different from what I had in mind, which was:
(1) Keep a reference to the base object in the buffer object, and
(2) Use the buffer API to fetch a fresh pointer from the base object each time it's needed.
Is there some reason that still wouldn't be safe enough?
That would work, be less intrusive, and allow all existing code to work unchanged. My only concern is that it does not go anywhere towards fixing the buffer interface itself. To my mind, the buffer object is fairly useless and I never use it - so I really don't care. However, I do have real world uses for the buffer interface. The most compelling is for async IO in the Windows world - I need to pass a buffer Windows will fill in the background, and the buffer interface provides the solution - except for the flaws that also drip down to the buffer object, and leaves us with this problem. Thus, my preference is to fix the buffer object by fixing the interface as much as possible. Here is a sketch of a solution, incorporating both Neil and Greg's ideas: * Type object gets a new flag - TP_HAS_BUFFER_INFO, corresponding to a new 'getbufferinfoproc' slot in the PyBufferProcs structure (note - a function pointer, not static flags as Neil suggested) * New function 'getbufferinfoproc' returns a bitmask - Py_BUFFER_FIXED is one (and currently the only) flag that can be returned. * New buffer functions PyObject_AsFixedCharBuffer, etc. These check the new flag (and a type lacking TP_HAS_BUFFER_INFO is assumed to *not* be fixed) * Buffer object keeps a reference to the existing object (as it does now). Its getbufferinfoproc delegates to the underlying object. * Buffer object *never* keeps a pointer to the buffer - only to the object. Functions like tp_hash always re-fetch the buffer on demand. The buffer returned by the buffer object is then guaranteed to be as reliable as the underlying object. (This may be a semantic issue with hash(), but conceptually seems fine. Potential solution here - add Py_BUFFER_READONLY as a buffer flag, then hash() semantics could do the right thing) After all that, I can't help noticing Greg's solution would be far less work <wink>, Mark.