[Python-ideas] ExternalMemory

[Kristján Valur Jónsson]

I find myself sometimes, when writing IO Code from C, wanting to pass memory that I have allocated internally and filled with data, to Python without copying it into a string object.
To this end, I have locally created a (2.x) method called PyBuffer_FromMemoryAndDestructor(), which is the same as PyBuffer_FromMemory() except that it will call a provided destructor function with an optional arg, to release the memory address given, when no longer in use.

First of all, I'd futilely  like to suggest this change for 2.x.  The existing PyBuffer_FromMemory() provides no lifetime management.
Second, the ByBuffer object doesn't support the new Py_buffer interface, so you can't really use this then, like putting a memoryview around it.  This is a fixable bug, otoh.

Thirdly, in py3k I think the situation is different.  There you would (probably, correct me if I'm wrong) emulate the old PyBuffer_FromMemory with a combination of the new PyBuffer_FromContiguous and a PyMemoryView_FromBuffer().  But this also does not allow any lifetime magement of the external memory.  So, for py3k, I'd actually like to extend the Memoryview object, and provide something like PyMemoryView_FromExternal() that takes an optional pointer to a "void destructor(void *arg, void *ptr)) and an (void *arg), to be called when the buffer is released.

K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101027/6afd605a/attachment.html>