ctypes And The WACAH Principle
eryk sun
eryksun at gmail.com
Wed Aug 10 06:12:52 EDT 2016
On Wed, Aug 10, 2016 at 1:45 AM, Lawrence D’Oliveiro
<lawrencedo99 at gmail.com> wrote:
>
> Ah, I wish... I had a look at the documentation for the buffer interface and memory views,
> and found no way to make use of them from pure Python. Basically, the paragraph I quoted
> above is wrong in just about every important detail.
ctypes classes have from_buffer and from_buffer_copy methods that use
the buffer protocol. For example, for a read-only buffer you can use
from_buffer_copy:
>>> b = bytes(b'1234')
>>> a = (ctypes.c_char * 3).from_buffer_copy(b, 1)
>>> a[:]
b'234'
from_buffer requires a writable buffer, a bit more explanation, and a
warning for Python 2. Switch to a bytearray to allow using
from_buffer:
>>> b = bytearray(b'1234')
>>> a = (ctypes.c_char * 3).from_buffer(b, 1)
>>> a[:]
b'234'
The memory is shared:
>>> b[:] = b[::-1]
>>> a[:]
b'321'
from_buffer in Python 3 uses a memoryview to reference the source
object and exported buffer, which it stores in the _objects dict of
the ctypes data instance:
>>> type(a._objects['ffffffff'])
<class 'memoryview'>
>>> a._objects['ffffffff'].obj
bytearray(b'4321')
Because of the existing export, the bytearray cannot be resized (up or down):
>>> del b[0]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
BufferError: Existing exports of data: object cannot be re-sized
On the other hand, from_buffer in Python 2 naively uses
PyObject_AsWriteBuffer and only keeps a reference to the source
object. Thus there's no guarantee that the memory will remain valid.
Let's demonstrate this with a segfault:
>>> b = bytearray(2**20)
>>> a = (ctypes.c_char * 2**20).from_buffer(b)
>>> b[-1] = 'z'
>>> a[-1]
'z'
>>> del b[:]
>>> a[-1]
Segmentation fault (core dumped)
More information about the Python-list
mailing list