Status of Numpy support in boost python - II
I was just re-reading these posts: Status of Numpy support in boost python http://web.archiveorange.com/archive/v/YJlerznoeRShLhlurQzC I wonder if there are any updates? Is ndarray working with current (1.45.0) boost?
----- Original Message ----
From: Neal Becker <ndbecker2@gmail.com> To: cplusplus-sig@python.org Sent: Mon, January 3, 2011 12:31:36 PM Subject: [C++-sig] Status of Numpy support in boost python - II
I was just re-reading these posts: Status of Numpy support in boost python http://web.archiveorange.com/archive/v/YJlerznoeRShLhlurQzC
I wonder if there are any updates?
Is ndarray working with current (1.45.0) boost?
Boost.Python's existing numpy array support has been working all the time. It sounds like you are expecting new functionality. Could you be more specific? FWIW: recently I've added C++ copies between numpy arrays and the scitbx array library. You can find the implementation in this directory: http://cctbx.svn.sourceforge.net/viewvc/cctbx/trunk/scitbx/array_family/boos... Look for numpy_bridge.hpp and numpy_bridge.cpp. You'll need something like this for other array libraries. Since there is no C++ "std" multi-dimensional array type it is difficult for Boost.Python to provide more support than it does currently. Ralf
On 01/03/2011 04:44 PM, Ralf W. Grosse-Kunstleve wrote:
----- Original Message ----
From: Neal Becker<ndbecker2@gmail.com> To: cplusplus-sig@python.org Sent: Mon, January 3, 2011 12:31:36 PM Subject: [C++-sig] Status of Numpy support in boost python - II
I was just re-reading these posts: Status of Numpy support in boost python http://web.archiveorange.com/archive/v/YJlerznoeRShLhlurQzC
I wonder if there are any updates?
Is ndarray working with current (1.45.0) boost? Boost.Python's existing numpy array support has been working all the time. It sounds like you are expecting new functionality. Could you be more specific?
One point that I'm particularly missing (and which I'm referring to in the thread cited above) is the ability to access raw pointers around. Here is a simple use-case illustrating my needs: I have a library that uses a separation between data types ("blocks") and algorithms, akin to the C++ standard library. I would like to wrap that library in a way that allows me to access and share data with other Python packages using NumPy (notably SciPy), without the need to copy. Right now that's impossible, since the only means to allocate NumPy arrays in boost.python is via the Python interpreter. In my own wrapper, I use the much richer PyArray_New. The returned pointer is then usable via PyArray_GETPTR<X>, allowing me to implement a "Block" type that I can bind my library to directly. Exposing such a rich API via boost.python obviously requires numpy/arrayobject.h to be available, at least during compilation of boost.python (or its numpy bindings), which may be considered inconvenient. I would be very happy for any such addition into boost.python, and would be happy to help implement it. Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin...
On 01/03/2011 01:44 PM, Ralf W. Grosse-Kunstleve wrote:
----- Original Message ----
From: Neal Becker<ndbecker2@gmail.com> To: cplusplus-sig@python.org Sent: Mon, January 3, 2011 12:31:36 PM Subject: [C++-sig] Status of Numpy support in boost python - II
I was just re-reading these posts: Status of Numpy support in boost python http://web.archiveorange.com/archive/v/YJlerznoeRShLhlurQzC
I wonder if there are any updates?
My low-level numpy library (basically a Boost.Python wrapper for the Numpy C-API) has received slow but steady updates. This is similar to the built-in Boost.Python numpy support, with a few extra features: - Raw-pointer access to Numpy arrays. - Constructing Numpy arrays from raw C++ pointers. - A C++ dtype object. - Tools to wrap C++ functors to behave like broadcasting numpy universal functions. This can be found in the boost sandbox: https://svn.boost.org/svn/boost/sandbox/numpy I have also built Boost.Python converters for my own ndarray library (see below) and Eigen 2. Right now the build system (scons-based) is tailored to my own needs, but it should be pretty easy for others to tweak as needed. Feel free to email me if you need help. I've included all of those in the ndarray package, though it should be possible to separate out the eigen bits and not use ndarray if you'd like. All that's here: http://code.google.com/p/ndarray/
Is ndarray working with current (1.45.0) boost?
Probably. I haven't tested 1.45, I compiled recently on 1.44. I have been doing most of my work using 1.40, but I don't think anything has changed in Boost.Python since then that would affect ndarray.
Boost.Python's existing numpy array support has been working all the time. It sounds like you are expecting new functionality. Could you be more specific?
<what Stefan Seefeld said> I think the low-level API I have in the sandbox fits this need, or at least provides some of the basics. I'd love to see that morph into something that we can all use to connect numpy to the C++ template library of our choice, since we'll all never agreed on those. Jim
<what Stefan Seefeld said>
I think the low-level API I have in the sandbox fits this need, or at least provides some of the basics. I'd love to see that morph into something that we can all use to connect numpy to the C++ template library of our choice,
Yes, that would be wonderful, indeed. Maybe a GSoC project? Ralf
On 01/03/2011 07:05 PM, Ralf W. Grosse-Kunstleve wrote:
<what Stefan Seefeld said> I think the low-level API I have in the sandbox fits this need, or at least provides some of the basics. I'd love to see that morph into something that we can all use to connect numpy to the C++ template library of our choice, Yes, that would be wonderful, indeed. Maybe a GSoC project?
Excellent idea. Let's see whether anyone is interested to pick it up (I'd be happy to mentor it). If not, perhaps we can make progress nonetheless. Stefan -- ...ich hab' noch einen Koffer in Berlin...
I also have made a small library that supports numpy. http://code.google.com/p/mds-utils/ Try it and look if it fits your needs. 2011/1/4 Stefan Seefeld <seefeld@sympatico.ca>
On 01/03/2011 07:05 PM, Ralf W. Grosse-Kunstleve wrote:
<what Stefan Seefeld said>
I think the low-level API I have in the sandbox fits this need, or at least provides some of the basics. I'd love to see that morph into something that we can all use to connect numpy to the C++ template library of our choice,
Yes, that would be wonderful, indeed. Maybe a GSoC project?
Excellent idea. Let's see whether anyone is interested to pick it up (I'd be happy to mentor it). If not, perhaps we can make progress nonetheless.
Stefan
--
...ich hab' noch einen Koffer in Berlin...
_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
-- Michele De Stefano http://www.micheledestefano.joomlafree.it <http://micheledestefano.xoom.it> http://code.google.com/p/mds-utils http://it.linkedin.com/in/micheledestefano
On 01/03/2011 04:46 PM, Stefan Seefeld wrote:
On 01/03/2011 07:05 PM, Ralf W. Grosse-Kunstleve wrote:
<what Stefan Seefeld said> I think the low-level API I have in the sandbox fits this need, or at least provides some of the basics. I'd love to see that morph into something that we can all use to connect numpy to the C++ template library of our choice, Yes, that would be wonderful, indeed. Maybe a GSoC project?
Excellent idea. Let's see whether anyone is interested to pick it up (I'd be happy to mentor it). If not, perhaps we can make progress nonetheless.
I'd be happy to help as well, though I don't really have the experience with boost development necessary to qualify as a mentor. I'll start by putting together a README file for the sandbox library over the next few days so others can get a better idea of where we might be starting from. Aside from direct data access (and the other things I have implemented already), what else should such a library do? One thing I can think of off the top of my head is automatic conversion for Numpy scalar types - I'm pretty sure someone else on this list has implemented that somewhere, but it's not present in the sandbox library. Jim
I'm interested in the ndarray http://code.google.com/p/ndarray package. I still need to figure out how to get it to compile against my customized boost installation in a non-standard location. Unfortunately, trying to build the 'test' results in ICE in gcc-4.5.1-4.fc14.x86_64 (reported upstream) g++ -o boost.python/ndarray/libs/python/ndarray/test/ndarray_mod.os -c -O2 -fno- strict-aliasing -g -pipe -Wall -fexceptions -fstack-protector --param=ssp- buffer-size=4 -m64 -mtune=generic -fPIC -fwrapv -fPIC -Wp,-D_FORTIFY_SOURCE=2 - DNDEBUG -D_GNU_SOURCE -Iinclude -I/usr/include/python2.7 - I/usr/lib64/python2.7/site-packages/numpy/core/include -I/usr/include/eigen2 boost.python/ndarray/libs/python/ndarray/test/ndarray_mod.cpp -I include In file included from include/ndarray/ArrayTraits.hpp:11:0, from include/ndarray/Array.hpp:11, from include/ndarray.hpp:9, from include/boost/python/ndarray/Array.hpp:5, from include/boost/python/ndarray/ndarray.hpp:4, from boost.python/ndarray/libs/python/ndarray/test/ndarray_mod.cpp:1: include/ndarray/ExpressionTraits.hpp: In instantiation of ‘ndarray::ExpressionTraits<ndarray::detail::BinaryOpExpression<ndarray::Array<double, 1, 1>, ndarray::Array<double, 1, 1>, ndarray::detail::EqualToTag::ScalarFunction<double, double>, 1> >’: include/ndarray/ExpressionBase.hpp:33:57: instantiated from ‘ndarray::ExpressionBase<ndarray::detail::BinaryOpExpression<ndarray::Array<double, 1, 1>, ndarray::Array<double, 1, 1>, ndarray::detail::EqualToTag::ScalarFunction<double, double>, 1> >’ include/ndarray/detail/BinaryOp.hpp:81:108: instantiated from ‘ndarray::detail::BinaryOpExpression<ndarray::Array<double, 1, 1>, ndarray::Array<double, 1, 1>, ndarray::detail::EqualToTag::ScalarFunction<double, double>, 1>’ boost.python/ndarray/libs/python/ndarray/test/ndarray_mod.cpp:44:36: instantiated from ‘bool acceptArray(const ndarray::Array<T, N, C>&) [with T = double, int N = 1, int C = 1]’ boost.python/ndarray/libs/python/ndarray/test/ndarray_mod.cpp:125:66: instantiated from here include/ndarray/ExpressionTraits.hpp:88:165: internal compiler error: Segmentation fault Jim Bosch wrote:
On 01/03/2011 01:44 PM, Ralf W. Grosse-Kunstleve wrote:
----- Original Message ----
From: Neal Becker<ndbecker2@gmail.com> To: cplusplus-sig@python.org Sent: Mon, January 3, 2011 12:31:36 PM Subject: [C++-sig] Status of Numpy support in boost python - II
I was just re-reading these posts: Status of Numpy support in boost python http://web.archiveorange.com/archive/v/YJlerznoeRShLhlurQzC
I wonder if there are any updates?
My low-level numpy library (basically a Boost.Python wrapper for the Numpy C-API) has received slow but steady updates. This is similar to the built-in Boost.Python numpy support, with a few extra features: - Raw-pointer access to Numpy arrays. - Constructing Numpy arrays from raw C++ pointers. - A C++ dtype object. - Tools to wrap C++ functors to behave like broadcasting numpy universal functions.
This can be found in the boost sandbox:
https://svn.boost.org/svn/boost/sandbox/numpy
I have also built Boost.Python converters for my own ndarray library (see below) and Eigen 2. Right now the build system (scons-based) is tailored to my own needs, but it should be pretty easy for others to tweak as needed. Feel free to email me if you need help.
I've included all of those in the ndarray package, though it should be possible to separate out the eigen bits and not use ndarray if you'd like. All that's here:
http://code.google.com/p/ndarray/
Is ndarray working with current (1.45.0) boost?
Probably. I haven't tested 1.45, I compiled recently on 1.44. I have been doing most of my work using 1.40, but I don't think anything has changed in Boost.Python since then that would affect ndarray.
Boost.Python's existing numpy array support has been working all the time. It sounds like you are expecting new functionality. Could you be more specific?
<what Stefan Seefeld said>
I think the low-level API I have in the sandbox fits this need, or at least provides some of the basics. I'd love to see that morph into something that we can all use to connect numpy to the C++ template library of our choice, since we'll all never agreed on those.
Jim
On 01/04/2011 05:34 AM, Neal Becker wrote:
I'm interested in the ndarray http://code.google.com/p/ndarray package.
I still need to figure out how to get it to compile against my customized boost installation in a non-standard location.
Unfortunately, trying to build the 'test' results in ICE in gcc-4.5.1-4.fc14.x86_64 (reported upstream)
Hmm. I haven't seen that, but I also haven't tried gcc 4.5 yet. Unfortunately, you've caught me in a busy week, but I'll make an effort to try gcc 4.5 (and boost 1.45) as soon as I can. Hopefully I can at least come up with a simple test case to help the gcc people out. In the meantime, if you just do "scons ndarray-test", do you still get the crash? That should exercise the same code that seems to be giving gcc trouble without any Python involvement. Jim
On 01/04/2011 11:35 AM, Jim Bosch wrote:
On 01/04/2011 05:34 AM, Neal Becker wrote:
I'm interested in the ndarray http://code.google.com/p/ndarray package.
I still need to figure out how to get it to compile against my customized boost installation in a non-standard location.
Unfortunately, trying to build the 'test' results in ICE in gcc-4.5.1-4.fc14.x86_64 (reported upstream)
Update: this is a known regression in gcc 4.5, and it looks like there's nothing I can do about it except work around it and wait for a fix to gcc. With that in mind, ndarray now disables some functionality (mathematical and comparison operators, but not including assignment and augmented assignment) when compiled with gcc 4.5. I have also updated the build system for both ndarray and Boost.Python.Numpy so one can specify additional include and library directories on the command line (see "scons --help" for more info). Neal, this should now be preferred to the SConstruct edit I suggested earlier. Jim
participants (5)
-
Jim Bosch -
Michele De Stefano -
Neal Becker -
Ralf W. Grosse-Kunstleve -
Stefan Seefeld