[Numpy-discussion] Best way to expose std::vector to be used with numpy

Ian Henriksen insertinterestingnamehere at gmail.com
Tue Oct 14 02:19:44 EDT 2014


On Mon, Oct 13, 2014 at 8:39 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

>
>
> On Mon, Oct 13, 2014 at 12:54 PM, Sebastian Berg <
> sebastian at sipsolutions.net> wrote:
>
>> On Mo, 2014-10-13 at 13:35 +0200, Daniele Nicolodi wrote:
>> > Hello,
>> >
>> > I have a C++ application that collects float, int or complex data in a
>> > possibly quite large std::vector. The application has some SWIG
>> > generated python wrappers that expose this vector to python. However,
>> > the standard way in which SWIG exposes the data is to create a touple
>> > and pass this to python, where it is very often converted to a numpy
>> > array for processing. Of course this is not efficient.
>> >
>> > I would like therefore to generate a smarter python interface. In python
>> > 3 I would without doubt go for implementing the buffer protocol, which
>> > enables seamless interfacing with numpy. However, I need to support also
>> > python 2 and there the buffer protocol is not as nice.
>>
>> Isn't the new buffer protocol in python 2.6 or 2.7 already? There is at
>> least a memoryview object in python 2, which maybe could be used to the
>> same effect?
>>
>
> No memoryview in python2.6, but the older buffer protocol it there. Is
> Boost Python an option?
>
> <snip>
>
> Chuck
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
Here's an idea on how to avoid the buffer protocol entirely.
It won't be perfectly optimal, but it is a simple solution that
may be good enough.

It could be that the shape and type inference and the Python
loops that are run when converting the large tuple to a NumPy
array are the main sticking points. In that case, the primary
bottleneck could be eliminated by looping and copying at the C level.
You could use Cython and just copy the needed information
manually into the array. Here's a rough outline of what the
Cython file might look like:

# distutils: language = c++
# distutils: libraries = (whatever shared library you need)
# distutils: library_dirs = (folders where the libraries are)
# distutils: include_dirs = (wherever the headers are)
# cython: boundscheck = False
# cython: wraparound = False

import numpy as np
from libcpp.vector cimport vector

# Here do the call to your C++ library
# This can be done by declaring the
# functions or whatever else you need in
# a cdef extern from "header file" block
# and then calling the functions to get
# the vector you need.

cdef int[::1] arr_from_vector(vector[int] v):
    cdef:
        int i
        int[::1] a = np.empty(v.size(), int)
    for i in xrange(v.size()):
        a[i] = v[i]
    return np.asarray(a)

def get_data():
    # Run your C++ computation here.
    return arr_from_vector(v)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20141014/1277b2e6/attachment.html>


More information about the NumPy-Discussion mailing list