Omit Py_buffer struct from Stable ABI for Python 3.2?
Currently [1], the implementation and the documentation for PEP 3118's Py_buffer struct don't line up (there's an extra field in the implementation that the PEP doesn't mention). Accordingly, Mark and I think it may be a good idea to leave this structure (and possibly related APIs) out of the stable ABI for the 3.2 release. I don't *think* it needs changing, but I'm not 100% certain until we finish working through the problem and realign the implementation and documentation. Applications and extension modules that use this interface would still work - they would just have to wait until 3.3 before they could consider migrating to the stable ABI. Regards, Nick. [1] http://bugs.python.org/issue10181 -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Wed, Jan 5, 2011 at 12:31 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Currently [1], the implementation and the documentation for PEP 3118's Py_buffer struct don't line up (there's an extra field in the implementation that the PEP doesn't mention).
I think there are actually two such fields: smalltable and obj. The need for obj is a little ugly: as far as I can tell, it's meaningless for a 3rd-party object that wants to export buffers---it's only really used by the memoryview object and by internal Python types. Mark
On Wed, 5 Jan 2011 12:55:55 +0000 Mark Dickinson <dickinsm@gmail.com> wrote:
On Wed, Jan 5, 2011 at 12:31 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Currently [1], the implementation and the documentation for PEP 3118's Py_buffer struct don't line up (there's an extra field in the implementation that the PEP doesn't mention).
I think there are actually two such fields: smalltable and obj.
The need for obj is a little ugly: as far as I can tell, it's meaningless for a 3rd-party object that wants to export buffers---it's only really used by the memoryview object and by internal Python types.
I don't think it's ugly. It's the only way to know which object exported a Py_buffer. Otherwise you have to track the information separately, which is quite a bit uglier (especially when in conjunction with PyArg_ParseTuple and friends). Regards Antoine.
On Wed, Jan 5, 2011 at 1:13 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Wed, 5 Jan 2011 12:55:55 +0000 Mark Dickinson <dickinsm@gmail.com> wrote:
The need for obj is a little ugly: as far as I can tell, it's meaningless for a 3rd-party object that wants to export buffers---it's only really used by the memoryview object and by internal Python types.
I don't think it's ugly. It's the only way to know which object exported a Py_buffer. Otherwise you have to track the information separately, which is quite a bit uglier (especially when in conjunction with PyArg_ParseTuple and friends).
Maybe I'm misunderstanding. What's the responsibility of a buffer export w.r.t. the obj field---i.e., what should 3rd party code be filling that obj field with in a call to getbuffer? It looks to me as though it's really the memoryview object that needs this information; that it doesn't belong in the Py_buffer struct. Isn't that what the 'base' field in PyMemoryViewObject in PEP 3118 was supposed to be for? Though I notice that that field is unused in the actual PyMemoryViewObject in Include/memoryobject.h. Mark
On Wed, Jan 5, 2011 at 2:03 PM, Mark Dickinson <dickinsm@gmail.com> wrote:
Maybe I'm misunderstanding. What's the responsibility of a buffer export w.r.t. the obj field---i.e., what should 3rd party code be
Grr. *buffer exporter*, not *buffer export*. Mark
On Wed, 5 Jan 2011 14:03:41 +0000 Mark Dickinson <dickinsm@gmail.com> wrote:
On Wed, Jan 5, 2011 at 1:13 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Wed, 5 Jan 2011 12:55:55 +0000 Mark Dickinson <dickinsm@gmail.com> wrote:
The need for obj is a little ugly: as far as I can tell, it's meaningless for a 3rd-party object that wants to export buffers---it's only really used by the memoryview object and by internal Python types.
I don't think it's ugly. It's the only way to know which object exported a Py_buffer. Otherwise you have to track the information separately, which is quite a bit uglier (especially when in conjunction with PyArg_ParseTuple and friends).
Maybe I'm misunderstanding. What's the responsibility of a buffer export w.r.t. the obj field---i.e., what should 3rd party code be filling that obj field with in a call to getbuffer?
It would let PyBuffer_FillInfo() do the job. If it doesn't want to, it must put itself in that field, and increment its reference count.
It looks to me as though it's really the memoryview object that needs this information;
No, anyone wanting to release a buffer without keeping separate track of the original object needs it. As I said, this also applies to users of PyArg_ParseTuple and friends ("s*", "y*" etc. format codes).
Isn't that what the 'base' field in PyMemoryViewObject in PEP 3118 was supposed to be for?
Perhaps, but practice (implementing "s*" etc.) suggested it was useful in other cases. That field ('base') is removed in 3.2. Regards Antoine.
On Thu, Jan 6, 2011 at 12:03 AM, Mark Dickinson <dickinsm@gmail.com> wrote:
Maybe I'm misunderstanding. What's the responsibility of a buffer export w.r.t. the obj field---i.e., what should 3rd party code be filling that obj field with in a call to getbuffer?
It should be a pointer to the object (with the reference count incremented appropriately). GetBuffer/ReleaseBuffer should actually manage it automatically, but I'd have to look at the code to make sure that is the case (and, if it isn't, there may be backwards compatibility implications in fixing it).
It looks to me as though it's really the memoryview object that needs this information; that it doesn't belong in the Py_buffer struct. Isn't that what the 'base' field in PyMemoryViewObject in PEP 3118 was supposed to be for? Though I notice that that field is unused in the actual PyMemoryViewObject in Include/memoryobject.h.
If nothing else, PyObject_ReleaseBuffer needs it - otherwise the function signature would need to include a separate argument to tell it who the buffer belongs to (so it can find the appropriate function pointer to call). The implementation makes sense (since every call to GetBuffer needs to be paired with a corresponding call to ReleaseBuffer, it makes sense to keep the object reference inside the Py_buffer struct), but the fact the documentation was never corrected suggests there are going to be plenty of broken implementations of the protocol kicking around, potentially even in the standard library. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Mark Dickinson, 05.01.2011 13:55:
On Wed, Jan 5, 2011 at 12:31 PM, Nick Coghlan wrote:
Currently [1], the implementation and the documentation for PEP 3118's Py_buffer struct don't line up (there's an extra field in the implementation that the PEP doesn't mention).
I think there are actually two such fields: smalltable and obj.
The need for obj is a little ugly: as far as I can tell, it's meaningless for a 3rd-party object that wants to export buffers---it's only really used by the memoryview object and by internal Python types.
Not at all. It's the reason why some of the buffer API functions could be changed to a simpler signature after earlier versions of the PEP had been written. Stefan
participants (4)
-
Antoine Pitrou -
Mark Dickinson -
Nick Coghlan -
Stefan Behnel