[C++-sig] [Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

Charles R Harris charlesr.harris at gmail.com
Wed Sep 12 13:56:23 CEST 2007


On 9/11/07, Alexander Schmolck <a.schmolck at gmx.net> wrote:
>
> Philip Austin <paustin at eos.ubc.ca> writes:
>
> > Alexander Schmolck writes:
> >
> >  > So my question is: might it make sense to use (a slightly wrapped)
> >  > numpy.ndarray, and if so is some code already floating around for
> that (on
> >  > first glance it seems like there's a bit of support for the obsolete
> Numeric
> >  > package in boost, but none for the newer numpy that supercedes it);
> if not is
> >  > my impression correct that making the existing code numpy compatible
> shouldn't
> >  > be too much work.
> >
> > Right, it should work more or less as is if you just do:
> >
> > set_module_and_type("numpy", "ArrayType");
>
> Ah, I guess that's the advantage of going via python, rather than calling
> the
> C-api directly (although I assume it must be rather costly).
>
> >
> > in the examples.  Some tests will fail because of numpy changes to
> > function signatures, etc.
> >
> > The current library doesn't wrap numpy.zeros, numpy.ones or
> > numpy.empty constructors, so the only way to construct an empty
> > is to pass the constructor a tuple and then resize.  Because
> > it (by design) doesn't include arrayobject.h, there's also no clean
> > way to get at the underlying data pointer to share the memory.
>
> Not being able to get at the data-pointer sounds like a show-stopper for
> this
> purpose -- I will almost certainly need to interface to exisitng C and C++
> code, and I do not intend to copy hundres of MB around unnecessarily.


You can get to the data pointer pretty easily through the buffer interface
if you just need contiguous memory. If you keep the dirty details on the
Python side, you can do something like:

#include <boost/python.hpp>
#include <stdexcept>

using namespace boost::python;

static const void* getReadBuffer(object obj, int size)
{
    int bufLen;
    void const *buffer;
    bool isReadBuffer = !PyObject_AsReadBuffer(obj.ptr(), &buffer, &bufLen);

    if (!isReadBuffer) {
        throw std::invalid_argument("Camera: buffer is not read buffer.");
    }
    if (bufLen != size) {
        throw std::invalid_argument("Camera: buffer is wrong size.");
    }
    return buffer;
}

You can certainly get fancier with shared pointers and stuff, but I can't
find the code at the moment.

I think it is a real shame that boost currently doesn't properly support
> numpy
> out of the box, although numpy has long obsoleted both numarray and
> Numeric
> (which is both buggy and completely unsupported). All the more so since
> writing multimedial or scientific extensions (in which numpy's array
> interface
> is very natural to figure prominently) would seem such an ideal use for
> boost.python, as soon as complex classes or compound structures that need
> to
> efficiently support several (primitive) datatypes are involved,
> boost.python
> could really play its strenghts compared to Fortran/C based extensions.
>
> I've since stumbled (in the numpy-list) upon
>
> <http://thread.gmane.org/gmane.comp.python.c++/11559/focus=11560>
>
> which seems like it could offer a great fit for my needs, but
> unfortunately
> it's not really documented and there's also no indication how ready for
> use it
> is -- I'd be interested to hear if anyone has an experience-report to
> offer;
> if not I guess I might just end up settling for std::vector for the time
> being, I need something workable soon, and it doesn't look like it'd be
> able
> to figure it out and verify that it works for me without a substantial
> time
> investment.
>
>
> > You can use helper functions like this, though:
> >
> > //Create a one-dimensional numpy array of length n and type t
> > boost::python::numeric::array makeNum(intp n, PyArray_TYPES
> t=PyArray_DOUBLE){
> >   object obj(handle<>(PyArray_FromDims(1, &n, t)));
> >   return extract<boost::python::numeric::array>(obj);
> > }
> >
> >
> http://www.eos.ubc.ca/research/clouds/software/pythonlibs/num_util/num_util_release2/Readme.html


Yep, that works.

If you need Cholesky for ublas, I've got templates for that. I've found
ublas quite efficient for small arrays when compiled as release code.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20070912/16a723ad/attachment.htm>


More information about the Cplusplus-sig mailing list