Buffers and pointers (Py3.4)

[Rob Gaddi]

So long as we're all feeling so concerned about speed lately...

I've got a piece of hardware that allows me to call a C library that 
calls an ioctl() that kicks off a DMA transfer and moves data between 
process memory and external hardware.  I would like a way to turn an 
arbitrary object implementing the buffer protocol into a pointer so that 
I can pass that pointer to the library to be passed to the hardware to 
stream my data.

The obvious question on your minds is why on earth am I doing this. 
Example case: my hardware is being used as a bridge to talk to a 16 
channel transient digitizer, acquiring data at 500 ksps per channel.  I 
would like my Python application to be able to acquire, say, 10M samples 
per channel.  I can have a list of 16 10M NumPy arrays, and every 5 ms I 
can request a single DMA transfer to grab all the new data that's been 
acquired and push it, zero-copy, into my arrays before the hardware 
FIFOs get full and I start losing data.

If I knew the buffers would always be NumPy arrays I could use 
ndarray.ctypes.data to get the pointer.  Always ctypes buffers, I could 
just use ctypes.addressof.  In either case I can flip them into 
memoryviews, but nothing seems to give me a pointer from that. 
Bytearrays, array.arrarys..., I've got nothin'.

If I were writing this as a C extension, getting that information from 
any buffer object would be trivial, but that changes my project from a 
pure Python wrapper using ctypes to a mixed language project with 
compile dependencies and mental headaches.

   buffertype = c_uint8 * size
   return addressof(buffertype.from_buffer(buf, offset))

works but is inefficient and woefully inelegant.  I saw some discussion 
on the ctypes bug tracker https://bugs.python.org/issue27274 about this 
nearly a year ago, but no actual action.

I really want to take all of this low-level horribleness, wrap it up 
into a higher-level API, and never be subject to the gory details again. 
  This pointer thing is about the last hangup.  Ideas?

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.