[Numpy-discussion] Best way to expose std::vector to be used with numpy
insertinterestingnamehere at gmail.com
Tue Oct 14 02:19:44 EDT 2014
On Mon, Oct 13, 2014 at 8:39 PM, Charles R Harris <charlesr.harris at gmail.com
> On Mon, Oct 13, 2014 at 12:54 PM, Sebastian Berg <
> sebastian at sipsolutions.net> wrote:
>> On Mo, 2014-10-13 at 13:35 +0200, Daniele Nicolodi wrote:
>> > Hello,
>> > I have a C++ application that collects float, int or complex data in a
>> > possibly quite large std::vector. The application has some SWIG
>> > generated python wrappers that expose this vector to python. However,
>> > the standard way in which SWIG exposes the data is to create a touple
>> > and pass this to python, where it is very often converted to a numpy
>> > array for processing. Of course this is not efficient.
>> > I would like therefore to generate a smarter python interface. In python
>> > 3 I would without doubt go for implementing the buffer protocol, which
>> > enables seamless interfacing with numpy. However, I need to support also
>> > python 2 and there the buffer protocol is not as nice.
>> Isn't the new buffer protocol in python 2.6 or 2.7 already? There is at
>> least a memoryview object in python 2, which maybe could be used to the
>> same effect?
> No memoryview in python2.6, but the older buffer protocol it there. Is
> Boost Python an option?
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
Here's an idea on how to avoid the buffer protocol entirely.
It won't be perfectly optimal, but it is a simple solution that
may be good enough.
It could be that the shape and type inference and the Python
loops that are run when converting the large tuple to a NumPy
array are the main sticking points. In that case, the primary
bottleneck could be eliminated by looping and copying at the C level.
You could use Cython and just copy the needed information
manually into the array. Here's a rough outline of what the
Cython file might look like:
# distutils: language = c++
# distutils: libraries = (whatever shared library you need)
# distutils: library_dirs = (folders where the libraries are)
# distutils: include_dirs = (wherever the headers are)
# cython: boundscheck = False
# cython: wraparound = False
import numpy as np
from libcpp.vector cimport vector
# Here do the call to your C++ library
# This can be done by declaring the
# functions or whatever else you need in
# a cdef extern from "header file" block
# and then calling the functions to get
# the vector you need.
cdef int[::1] arr_from_vector(vector[int] v):
int[::1] a = np.empty(v.size(), int)
for i in xrange(v.size()):
a[i] = v[i]
# Run your C++ computation here.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion