Numpy and C++ integration...
Dear Numpy Experts- I find myself working with Numpy arrays and wanting to access *simple* C++ functions for time series returning the results to Numpy. As I am a relatively new user of Python/Numpy, the number of paths to use in incorporating C++ code into one's scripts is daunting. I've attempted the Weave app but can not get past the examples. I've also looked at all the other choices out there such as Boost, SIP, PyInline, etc. Any trailheads for the simplest approach (assuming a very minimal understanding of C++) would be much appreciated. At this point, I can't release the code however for review. Thank you. -- Vince Fulco
On Mon, Feb 04, 2008 at 11:02:29AM -0500, Vince Fulco wrote:
Any trailheads for the simplest approach
I find ctypes very easy to understand. See http://www.scipy.org/Cookbook/Ctypes for simple instructions. HTH, Gaël
Dear Mr. Fulco , This may not be exactly what you want to do, but I would recommend using the C API and then calling your C++ programs from there (where interface functions to the C++ code is compiled in the extern "C" {, } block. I will be doing this soon with my own project. Why? Because the C interface is doable and, I think, simple enough that it is better to take the Python to C++ in two steps. Anyway, worth a look. So here are two links that show how to use the C API: http://www.scipy.org/Cookbook/C_Extensions - A short intro, this also has documentation links http://www.scipy.org/Cookbook/C_Extensions/NumPy_arrays?highlight=%28%28----... - This is an article I wrote last year for the SciPy.org site and I go into a lot of detail with a lot of examples on how you pass and handle Numpy arrays. I think it is (mostly) right and works well for me. One warning (which I also talk about in my tutorial) is to make sure your NumPy arrays are "Continguous", i.e. the array components are in order in one memory block. That makes things easier on the C/C++ side. --- Vince Fulco <vfulco1@gmail.com> wrote:
Dear Numpy Experts- I find myself working with Numpy arrays and wanting to access *simple* C++ functions for time series returning the results to Numpy. As I am a relatively new user of Python/Numpy, the number of paths to use in incorporating C++ code into one's scripts is daunting. I've attempted the Weave app but can not get past the examples. I've also looked at all the other choices out there such as Boost, SIP, PyInline, etc. Any trailheads for the simplest approach (assuming a very minimal understanding of C++) would be much appreciated. At this point, I can't release the code however for review. Thank you.
-- Vince Fulco
-- Lou Pecora, my views are my own. ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
2008/2/4, Lou Pecora <lou_boog2000@yahoo.com>:
Dear Mr. Fulco ,
This may not be exactly what you want to do, but I would recommend using the C API and then calling your C++ programs from there (where interface functions to the C++ code is compiled in the extern "C" {, } block. I will be doing this soon with my own project. Why? Because the C interface is doable and, I think, simple enough that it is better to take the Python to C++ in two steps. Anyway, worth a look. So here are two links that show how to use the C API:
http://www.scipy.org/Cookbook/C_Extensions - A short intro, this also has documentation links
http://www.scipy.org/Cookbook/C_Extensions/NumPy_arrays?highlight=%28%28----... - This is an article I wrote last year for the SciPy.org site and I go into a lot of detail with a lot of examples on how you pass and handle Numpy arrays. I think it is (mostly) right and works well for me.
One warning (which I also talk about in my tutorial) is to make sure your NumPy arrays are "Continguous", i.e. the array components are in order in one memory block. That makes things easier on the C/C++ side.
Whatever solution you choose (Boost.Python, ...), you will have to use the Numpy C API at least a little bit. So Travis' book is a good start. As Gaël told you, you can use ctypes if you wrap manually every method with a C function and recreate the class in Python. This can be avoided, but you'll have to use more powerful tools. I would advice SWIG (see my blog for some examples with C++ and SWIG). Matthieu -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher
--- Matthieu Brucher <matthieu.brucher@gmail.com> wrote:
Whatever solution you choose (Boost.Python, ...), you will have to use the Numpy C API at least a little bit. So Travis' book is a good start. As Gaël told you, you can use ctypes if you wrap manually every method with a C function and recreate the class in Python. This can be avoided, but you'll have to use more powerful tools. I would advice SWIG (see my blog for some examples with C++ and SWIG).
Matthieu
Ah, yes, I will also recommend Travis' book. -- Lou Pecora, my views are my own. ____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
I have a variety of experiments that I put in this mercurial repo: https://nbecker.dyndns.org/hg/ The primary aim of this is to reuse c++ code written to a generic container interface, with numpy.
Neal Becker wrote:
I have a variety of experiments that I put in this mercurial repo: https://nbecker.dyndns.org/hg/
The primary aim of this is to reuse c++ code written to a generic container interface, with numpy.
Neal, I'd love to hear more about this. Do you have a two paragraph description of what you're up to? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Christopher Barker wrote:
Neal Becker wrote:
I have a variety of experiments that I put in this mercurial repo: https://nbecker.dyndns.org/hg/
The primary aim of this is to reuse c++ code written to a generic container interface, with numpy.
Neal,
I'd love to hear more about this. Do you have a two paragraph description of what you're up to?
I need to update it, but here is a short doc: https://nbecker.dyndns.org/misc/design.pdf If you look at the hg repo, you will see a few interesting exercises. accumulator shows the idea of making 1-d numpy arrays usable as containers compatible with boost::range. numpy_iter.hpp has most of the work. This has: n-dim wrapper for numpy array, and iterator to go with it. Since the n-d iter concept is not too well defined, I haven't worked on this much. 1-dim wrapper and iterator to go with it. numpy is run-time polymorphic, which the c++ code I want to use is compile-time. misc has some tests of dispatching based on types. num2.cc has some tests of creating numpy arrays. limit.cc has a little test of ufunc.
On Feb 4, 2008, at 9:39 AM, Matthieu Brucher wrote:
This can be avoided, but you'll have to use more powerful tools. I would advice SWIG (see my blog for some examples with C++ and SWIG).
Note that a lot of work has been done to bridge between numpy and swig. There is a swig interface file for numpy in numpy/doc/swig, along with documentation on how to use it. It is largely geared towards generating wrappers to C/C++ functions that take pointers to arrays. If your data is encapsulated within a C++ class, then it gets more complicated. ** Bill Spotz ** ** Sandia National Laboratories Voice: (505)845-0170 ** ** P.O. Box 5800 Fax: (505)284-0154 ** ** Albuquerque, NM 87185-0370 Email: wfspotz@sandia.gov **
Lou Pecora wrote:
I would recommend using the C API
I would recommend against this -- there is a lot of code to write in extensions to make sure you do reference counting, etc, and it is hard to get right. Much of it is also boiler-plate code, so it makes more sense to have that code auto-generated. There are just too many good tools to do this for you to do it by hand. The problem is that there is an embarrassment of riches -- if only one or tow of the C/C++ interface tools were out there, it would be a whole lot easier to choose! My take: ctypes -- best if you have dll-type interface already defined, and particularly if there are a smallish number of functions you want to call. Can it call C++ directly at all? pyrex -- best if you want to implement some custom functions in C from scratch. Also pretty good for calling external C code. Only supports calling C++ code that's not too fancy -- i.e. stuff that can be called from C -- pyrex has no explicit support for C++ SWIG -- best if you have a lot of code to wrap that shares similar interfaces - once you write the typemaps, the rest is automatic. Also best choice if you want to support more than one scripting language, or you want to integrate with other packages built with SWIG (wxPython, GDAL, VTK, ...). Bill's numpy-swig typemaps make it easy to deal with classic C-style pointers to data blocks. It also comes with built-in wrappers for std:vector, though not numpy integration for those. SIP -- built for pyQT -- folks seem to like it. I don't know if anyone has done anything for numpy with it. Boost::python -- best for writing custom extensions in C++ -- also can be used for interfacing with legacy C++. There were boost array classes for numpy -- are these being maintained? Any of these can do that job -- so it's hard to choose, but maybe the above helps focus your search. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Feb 4, 2008, at 1:05 PM, Christopher Barker wrote:
Boost::python -- best for writing custom extensions in C++ -- also can be used for interfacing with legacy C++. There were boost array classes for numpy -- are these being maintained?
There are boost array classes for Numeric, and *talk* of upgrading them to numpy, but it hasn't happened yet, to my knowledge. ** Bill Spotz ** ** Sandia National Laboratories Voice: (505)845-0170 ** ** P.O. Box 5800 Fax: (505)284-0154 ** ** Albuquerque, NM 87185-0370 Email: wfspotz@sandia.gov **
Bill Spotz wrote:
On Feb 4, 2008, at 1:05 PM, Christopher Barker wrote:
Boost::python -- best for writing custom extensions in C++ -- also can be used for interfacing with legacy C++. There were boost array classes for numpy -- are these being maintained?
There are boost array classes for Numeric, and *talk* of upgrading them to numpy, but it hasn't happened yet, to my knowledge.
I've updated and tested (but not uploaded) our num_util helper functions (http://www.eos.ubc.ca/research/clouds/software/pythonlibs/num_util/num_util_...) for python 2.5.1, numpy 1.0.5.dev4731, boost_1_34_1, Centos 5.1 64bit on Athlon64. Anyone that's interested is welcome to contact me for a tarfile. I plan to switch the underlying array object from boost::python::numeric::array to Neal's numpy_array, but that's not likely to happen before April -- Phil
--- Christopher Barker <Chris.Barker@noaa.gov> wrote:
Lou Pecora wrote:
I would recommend using the C API
I would recommend against this -- there is a lot of code to write in extensions to make sure you do reference counting, etc, and it is hard to get right.
Well, fair enough to some extent, but I didn't find it so hard after I did a few. I will speak for myself here. The reason I went to the C API is because I tried several of the routes you suggest and I could not get any of them to work. And you're right, the C API is boilerplate. That also argues for using it. So, for those looking for speed up through some external C or C++ code, I would say (trying to be fair here), try what Chris recommends below, if you want, but IMHO, none of it is trivial. If you get it to work, great. If not, you have the fall back of the C API.
ctypes pyrex SWIG SIP Boost::python
I tried all of these except SIP and got nowhere. So maybe others will be a lot smarter than I.
Christopher Barker, Ph.D. Oceanographer
-- Lou Pecora, my views are my own. ____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
On Mon, Feb 04, 2008 at 12:49:58PM -0800, Lou Pecora wrote:
So, for those looking for speed up through some external C or C++ code, I would say (trying to be fair here), try what Chris recommends below, if you want, but IMHO, none of it is trivial. If you get it to work, great. If not, you have the fall back of the C API.
Honestly, I found ctypes trivial, using http://www.scipy.org/Cookbook/Ctypes as a reference, and simply copying the code. I started by copying the code, made sure it worked, and I understood how to modify it, than adapted it to my problem. Gaël
For comparison of ctypes and SWIG wrappers of a simple C++ codebase, feel free to take a look at the code for scikits.ann (http://scipy.org/scipy/scikits/wiki/AnnWrapper). The original wrapper was written using SWIG and the numpy typemaps. Rob Hetland has coded an almost-the-same API wrapper using ctypes which is linked from the above wiki page. Perhaps it will help the OP to see a similar project side-by-side. Barry On Feb 4, 2008 1:11 PM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
On Mon, Feb 04, 2008 at 12:49:58PM -0800, Lou Pecora wrote:
So, for those looking for speed up through some external C or C++ code, I would say (trying to be fair here), try what Chris recommends below, if you want, but IMHO, none of it is trivial. If you get it to work, great. If not, you have the fall back of the C API.
Honestly, I found ctypes trivial, using http://www.scipy.org/Cookbook/Ctypes as a reference, and simply copying the code. I started by copying the code, made sure it worked, and I understood how to modify it, than adapted it to my problem.
Gaël
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
On Feb 4, 2008 9:05 PM, Christopher Barker <Chris.Barker@noaa.gov> wrote:
Lou Pecora wrote:
I would recommend using the C API
I would recommend against this -- there is a lot of code to write in extensions to make sure you do reference counting, etc, and it is hard to get right.
Much of it is also boiler-plate code, so it makes more sense to have that code auto-generated.
There are just too many good tools to do this for you to do it by hand.
The problem is that there is an embarrassment of riches -- if only one or tow of the C/C++ interface tools were out there, it would be a whole lot easier to choose! My take:
ctypes -- best if you have dll-type interface already defined, and particularly if there are a smallish number of functions you want to call. Can it call C++ directly at all?
pyrex -- best if you want to implement some custom functions in C from scratch. Also pretty good for calling external C code. Only supports calling C++ code that's not too fancy -- i.e. stuff that can be called from C -- pyrex has no explicit support for C++
SWIG -- best if you have a lot of code to wrap that shares similar interfaces - once you write the typemaps, the rest is automatic. Also best choice if you want to support more than one scripting language, or you want to integrate with other packages built with SWIG (wxPython, GDAL, VTK, ...). Bill's numpy-swig typemaps make it easy to deal with classic C-style pointers to data blocks. It also comes with built-in wrappers for std:vector, though not numpy integration for those.
SIP -- built for pyQT -- folks seem to like it. I don't know if anyone has done anything for numpy with it.
Boost::python -- best for writing custom extensions in C++ -- also can be used for interfacing with legacy C++. There were boost array classes for numpy -- are these being maintained?
Any of these can do that job -- so it's hard to choose, but maybe the above helps focus your search.
-Chris Hi, Which of these can automatically wrap (and "overload" !!) function templates ? I have many "helper" functions which can operate equally on many -- if not all -- number types. As examples I would like to mention my favorite function that I call "mmms" which calculates min,max,mean,std.dev. in one go. I use SWIG with my own typemaps which (by now) might be similar to what is in numpy (written by Bill). They can handle uint8, int16, uint16, uint32, int32, float32,float64. A C-preprocessor function instantiates the function for each type and another (SWIG command) overloads it for python. They work without any memory-copy (!!) *if* the data is C-contiguous.
Can ctypes do this ? Thanks, -Sebastian
Gael Varoquaux wrote:
On Tue, Feb 05, 2008 at 09:15:29AM +0100, Sebastian Haase wrote:
Can ctypes do this ?
No. Ctypes is only a way of loading C (and not C++) libraries in Python. That makes it very simple, but not very powerful.
I would not call ctypes not very powerful :) For sure you cannot do the same way as swig does, but you could imagine some automatic scheme to solve Sebastian's problem. Typically, having a C wrapper automatically generated from the C++ headers, you could use the ctypes code generator, and you have something almost automatic (with a bit some boilerplate code in python, maybe). cheers, David
On Feb 5, 2008 11:23 AM, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
Gael Varoquaux wrote:
On Tue, Feb 05, 2008 at 09:15:29AM +0100, Sebastian Haase wrote:
Can ctypes do this ?
No. Ctypes is only a way of loading C (and not C++) libraries in Python. That makes it very simple, but not very powerful.
I would not call ctypes not very powerful :) For sure you cannot do the same way as swig does, but you could imagine some automatic scheme to solve Sebastian's problem.
Typically, having a C wrapper automatically generated from the C++ headers, you could use the ctypes code generator, and you have something almost automatic (with a bit some boilerplate code in python, maybe).
cheers,
Also feel free to extend this wiki: http://wiki.cython.org/WrappingCorCpp I use Cython, mostly for the same reasons Gael is using ctypes - it's trivial. Ondrej
On Tue, Feb 05, 2008 at 11:48:38AM +0100, Ondrej Certik wrote:
I use Cython, mostly for the same reasons Gael is using ctypes - it's trivial.
Actually, when I want to do something really trivial, I use scipy.weave.inline ( see http://scipy.org/PerformancePython for an example of scipy.weave.inline use). Of course it doesn't work when linking to external libraries, but to accelerate a for loop, it's great. Gaël
On Feb 5, 2008 11:52 AM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
On Tue, Feb 05, 2008 at 11:48:38AM +0100, Ondrej Certik wrote:
I use Cython, mostly for the same reasons Gael is using ctypes - it's trivial.
Actually, when I want to do something really trivial, I use scipy.weave.inline ( see http://scipy.org/PerformancePython for an example of scipy.weave.inline use). Of course it doesn't work when linking to external libraries, but to accelerate a for loop, it's great.
Yep. The power of Cython/Pyrex strategy is that you can work as I described here: http://ondrej.certik.cz/development/ Ondrej
On Feb 5, 2008 9:21 AM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
On Tue, Feb 05, 2008 at 09:15:29AM +0100, Sebastian Haase wrote:
Can ctypes do this ?
No. Ctypes is only a way of loading C (and not C++) libraries in Python. That makes it very simple, but not very powerful.
Gaël (sorry, this email got stuck in moderation, because I used the wrong sender address)
Thanks fr the reply. How about "manual" overloading. I mean, if -- for example -- I have two functions mmms_b and mmms_i in C, I could still use ctypes; could I then "merge" them into one python function, which "re-routes" depending on the argument dtype !? This is what SWIG must be doing internally -- right ?! Numpy/ctypes could come with such "re-routing helper meta-function" (decorators?) out-of-the-box... Thanks, -Sebastian
On Tue, Feb 05, 2008 at 11:48:37AM +0100, Sebastian Haase wrote:
Thanks fr the reply. How about "manual" overloading. I mean, if -- for example -- I have two functions mmms_b and mmms_i in C, I could still use ctypes; could I then "merge" them into one python function, which "re-routes" depending on the argument dtype !?
Yes, that's exactly what I do (except I rarely use C++, so I reroute to different C functions). It doesn't scale well, but it's OK if you have only a few functions to worry about. Gaël
On Tue, Feb 05, 2008 at 11:48:37AM +0100, Sebastian Haase wrote:
Thanks fr the reply. How about "manual" overloading. I mean, if -- for example -- I have two functions mmms_b and mmms_i in C, I could still use ctypes; could I then "merge" them into one python function, which "re-routes" depending on the argument dtype !?
Yes, that's exactly what I do (except I rarely use C++, so I reroute to different C functions). It doesn't scale well, but it's OK if you have only a few functions to worry about. It does not scale well if you do it manually, but I don't see any reason why it cannot be automated. ctypes main developer, Thomas Heller, has developed a code generator which parses C headers (which uses gcc-xml,
Gael Varoquaux wrote: that is can parse anything you can throw at gcc) and gives you ctypes-compatible code. It can be used on windows.h, which is the most horrendous/biggest header I can think of :) When using C++, you would need to find a way to generate C headers (needed anyway because you cannot dynamically load C++ code and used functions with C++ linkage, at least in a cross platform way). I have never used the code generator in such a way (handling multiple types), but I have used it to wrap win32 functions for sound IO, and it works pretty well. cheers, David
This is what SWIG must be doing internally -- right ?!
Yes, it is with an additional typemap that checks the type of the data. I don't think that it is a good idea for numpy to add such multi-dispatching, it is not its job. There are a lot of ways to do it, and besides it would be very cumbersome to have something specific to Numpy (that is only for Numpy's types), because when ctypes will be used with something else than Numpy, it would be a nightmare. The only thing you have to do to have your own dispatcher is to wrap the C functions in a Python function that will call the good function. It is very straightforward, so you might think that this solution could be integarted into Numpy but : - for such simple code, it is not very useful, a wiki entry can do better - if you have a double dispatch to do, you would have to implement it yourself, so you would have to check the code (or the wiki entry) and you would send a mail to include a double dispatcher to Numpy - etc for a triple dispatcher, four Matthieu -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher
Hmmm... last time I tried ctypes it seemed pretty Windows oriented and I got nowhere. But enough people have said how easy it is that I'll give it another try. Believe me, I'd be happy to be wrong and find a nice easy way to pass NumPy arrays and such. Thanks. -- Lou Pecora --- Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
On Tue, Feb 05, 2008 at 09:15:29AM +0100, Sebastian Haase wrote:
Can ctypes do this ?
No. Ctypes is only a way of loading C (and not C++) libraries in Python. That makes it very simple, but not very powerful. Gaël
____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
On Tue, Feb 05, 2008 at 06:45:25AM -0800, Lou Pecora wrote:
Hmmm... last time I tried ctypes it seemed pretty Windows oriented and I got nowhere. But enough people have said how easy it is that I'll give it another try.
I don't use windows much. One thing I liked about ctypes when I used it, was that what I found it pretty easy to get working on both Linux and Windows. Gaël
--- Gael Varoquaux <gael.varoquaux@normalesup.org> wrote: Re: ctypes
I don't use windows much. One thing I liked about ctypes when I used it, was that what I found it pretty easy to get working on both Linux and Windows.
Gaël
I got ctypes to install easily on Mac OS X 10.4.11 and it passed the test using python setup.py test. Now I have to find some examples on using it and learn to compile shared libraries (.so type I guess). -- Lou Pecora, my views are my own. ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
I got ctypes installed and passing its own tests. But I cannot get the shared library to load. I am using Mac OS X 10.4.11, Python 2.4 running through the Terminal. I am using Albert Strasheim's example on http://scipy.org/Cookbook/Ctypes2 except that I had to remove the defined 'extern' for FOO_API since the gcc compiler complained about two 'externs' (I don't really understand what the extern does here anyway). My make file for generating the library is simple, # ---- Link --------------------------- test1ctypes.so: test1ctypes.o test1ctypes.mak gcc -bundle -flat_namespace -undefined suppress -o test1ctypes.so test1ctypes.o # ---- gcc C compile ------------------ test1ctypes.o: test1ctypes.c test1ctypes.h test1ctypes.mak gcc -c test1ctypes.c -o test1ctypes.o This generates the file test1ctypes.so. But when I try to load it import numpy as N import ctypes as C _test1 = N.ctypeslib.load_library('test1ctypes', '.') I get the error message, OSError: dlopen(/Users/loupecora/test1ctypes.dylib, 6): image not found I've been googling for two hours trying to find the problem or other examples that would give me a clue, but no luck. Any ideas what I'm doing wrong? Thanks for any clues. -- Lou Pecora, my views are my own. ____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs
Lou Pecora wrote:
I got ctypes installed and passing its own tests. But I cannot get the shared library to load. I am using Mac OS X 10.4.11, Python 2.4 running through the Terminal.
I am using Albert Strasheim's example on http://scipy.org/Cookbook/Ctypes2 except that I had to remove the defined 'extern' for FOO_API since the gcc compiler complained about two 'externs' (I don't really understand what the extern does here anyway).
My make file for generating the library is simple,
# ---- Link --------------------------- test1ctypes.so: test1ctypes.o test1ctypes.mak gcc -bundle -flat_namespace -undefined suppress -o test1ctypes.so test1ctypes.o
# ---- gcc C compile ------------------ test1ctypes.o: test1ctypes.c test1ctypes.h test1ctypes.mak gcc -c test1ctypes.c -o test1ctypes.o
This generates the file test1ctypes.so. But when I try to load it
import numpy as N import ctypes as C
_test1 = N.ctypeslib.load_library('test1ctypes', '.')
I get the error message,
OSError: dlopen(/Users/loupecora/test1ctypes.dylib, 6): image not found
I've been googling for two hours trying to find the problem or other examples that would give me a clue, but no luck.
Any ideas what I'm doing wrong? Thanks for any clues.
Well, it's looking for test1ctypes.dylib, which I guess is a MacOSX shared library? Meanwhile, you made a test1ctypes.so, which is why it can't find it. You could try using this instead: _test1 = N.ctypeslib.load_library('test1ctypes.so', '.') or try to get gcc to make a test1ctypes.dylib. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma
Well, it's looking for test1ctypes.dylib, which I guess is a MacOSX shared library? Meanwhile, you made a test1ctypes.so, which is why it can't find it. You could try using this instead:
_test1 = N.ctypeslib.load_library('test1ctypes.so', '.')
or try to get gcc to make a test1ctypes.dylib.
Ryan
Thanks, Ryan. You were on the right track. I changed the name of the file in the load_library call to test1ctypes.so and I had to put in the full path to the file as the second argument. The default path was to my home directory. I could probably change paths with a python os call, too. Anyway, IT WORKED! How 'bout that? One simple example down and now on to more complex things. Thanks, again. -- Lou Pecora, my views are my own. ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
Vince Fulco <vfulco1 <at> gmail.com> writes:
Dear Numpy Experts- I find myself working with Numpy arrays and wanting to access *simple* C++ functions for time series returning the results to Numpy. As I am a relatively new user of Python/Numpy, the number of paths to use in incorporating C++ code into one's scripts is daunting. I've attempted the Weave app but can not get past the examples. I've also looked at all the other choices out there such as Boost, SIP, PyInline, etc. Any trailheads for the simplest approach (assuming a very minimal understanding of C++) would be much appreciated. At this point, I can't release the code however for review. Thank you.
We have created a small Python module Instant (www.fenics.org/instant) on top of SWIG, which makes integration of C/C++ and NumPy arrays easy in some cases. Its use: import numpy from instant import inline_with_numpy c_code = """ double sum (int n1, double* array1){ double tmp = 0.0; for (int i=0; i<n1; i++) { tmp += array1[i]; } return tmp; } """ sum_func = inline_with_numeric(c_code, arrays = [['n1', 'array1']]) a = numpy.arange(10000000); a = numpy.sin(a) sum1 = sum_func(a) Kent
Dear Vince, You probably have heard better solutions but I think what I do works and is simple to learn. When I need to call C++ code from Python, I write a wrapper extern "C" function that calls the C++ function that returns the result. Then I just use ctypes to call the extern "C" function from Python. C++/C: extern "C" { double *get_result(double *input, int n) { return CPlusPlusFunction::GetResult(input, n); } } Python: import ctypes mylib = ctypes.CDLL('libmylib') def get_result(A): return mylib.get_result(input.ctypes.data) I hope this helps. Damian
Vince Fulco <vfulco1 <at> gmail.com> writes:
Dear Numpy Experts- I find myself working with Numpy arrays and wanting to access *simple* C++ functions for time series returning the results to Numpy. As I am a relatively new user of Python/Numpy, the number of paths to use in incorporating C++ code into one's scripts is daunting. I've attempted the Weave app but can not get past the examples. I've also looked at all the other choices out there such as Boost, SIP, PyInline, etc. Any trailheads for the simplest approach (assuming a very minimal understanding of C++) would be much appreciated. At this point, I can't release the code however for review. Thank you.
On Tue, Feb 05, 2008 at 12:16:02PM -0600, Kent-Andre Mardal wrote:
We have created a small Python module Instant (www.fenics.org/instant) on top of SWIG, which makes integration of C/C++ and NumPy arrays easy in some cases.
Hello, Thank you for posting about instant. I think it looks like a great idea and hope to try it out soon. I noticed that you are distributing under the GPL. Would you consider releasing it under a more permissive license? Some rationale is given here: http://www.scipy.org/License_Compatibility Best Regards, Glen mabey
How does Instant compare to scipy.weave !? -Sebastian Haase On Feb 5, 2008 11:26 PM, Glen W. Mabey <Glen.Mabey@swri.org> wrote:
On Tue, Feb 05, 2008 at 12:16:02PM -0600, Kent-Andre Mardal wrote:
We have created a small Python module Instant (www.fenics.org/instant) on top of SWIG, which makes integration of C/C++ and NumPy arrays easy in some cases.
Hello,
Thank you for posting about instant. I think it looks like a great idea and hope to try it out soon.
I noticed that you are distributing under the GPL.
Would you consider releasing it under a more permissive license?
Some rationale is given here:
http://www.scipy.org/License_Compatibility
Best Regards, Glen mabey
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
participants (16)
-
Barry Wark -
Bill Spotz -
Christopher Barker -
Damian Eads -
David Cournapeau -
Gael Varoquaux -
Glen W. Mabey -
Kent-Andre Mardal -
Lou Pecora -
Matthieu Brucher -
Neal Becker -
Ondrej Certik -
Phil Austin -
Ryan May -
Sebastian Haase -
Vince Fulco