I'm concerned about the direction that this PEP seems to be going. The original proposal was borderline too complicated IMO, and now it seems headed in the direction of more complexity.
Also, it seems that there are three different goals getting conflated here. None are bad, but they don't and probably shouldn't, all be addressed by the same PEP.
- Allowing producers and consumers of blocks of data to share blocks efficiently. This is half of what the original PEP proposed.
- Describing complex data types at the c-level. This is the other half of the PEP.
- Things that act like arrays, but have different storage methods. This details of this still seem pretty vague, but to the extent that I can figure them out, it doesn't seem useful or necessary to tie this into the rest of the array interface PEP. For example, "array_interface->get_block_from_slice()" has been mentioned. Why that instead of "PyObject_AsExtendedBuffer(PyObject_GetItem(index), ....)". I'll stop here, till I see some more details of what people have in mind, but at this point, I think that alternative memory models are a different problem that should be addressed separately.
Sadly, I'm leaving town shortly and I'm running out of time, so I'll have to leave my objections in this somewhat vague state.
Oh, the way that F. Lundh plans to expose PIL's data a chunk at a time is mentioned in this python-dev summary:
It doesn't seem necessary to have special support for this; all that is necessary is for the object returned by acquire_view to support the extended array protocol.