passing a C array to embedded Python from C code
Hello, I'm working on seismic processing software called Globe Claritas. The core is written in C and a bit of Fortran. I would like to embed Python in this software, so a user can use Python code to manipulate the seismic data. This would give our users all the power of Python and NumPy and any external modules they may want to use. I was thinking I could use the Python C API to initialize Python and run the code supplied by the user. I'm thinking I could create a NumPy array that uses the same C array (1d, 2d, or 3d) that our program is using instead of copying the memory. Perhaps using the NumPy API function PyArray_SimpleNewFromData() or something similar (similar to this posting: http://blog.enthought.com/?p=62). I would then pass this NumPy array as an object to the Python code, which it could manipulate. The C array is an array of floating point data samples for each seismic trace, so hopefully well suited to this. At this point, I'm thinking the python code wouldn't need to return any objects because it would be modifying the seismic data (and trace headers) in memory. Does this approach make sense? Is there a better way to go about it? Maybe calling a custom module from the Python code that does the C array to NumPy translation using Cython/pyrex/swig/etc. Would it be possible to use the same C arrays from here without copying them? Is there a more obvious way to do it that I might be missing? Does anyone have examples of creating a NumPy array in C, and passing it to a Python instance? That would be a huge help, I have no experience with the C or NumPy APIs. Thanks, Chris LeBlanc
Does this approach make sense? Is there a better way to go about it? Maybe calling a custom module from the Python code that does the C array to NumPy translation using Cython/pyrex/swig/etc. Would it be possible to use the same C arrays from here without copying them?
Hi, Your case seems to fit the array interface. The goal is to create a C structure with some additional information that Numpy can understand, and then your array will be treated as a Numpy array. If you can follow a French tutorial, you can go on http://matthieu-brucher.developpez.com/tutoriels/python/swig-numpy/#LV to have a skeletton for your issue. Matthieu -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher
Matthieu Brucher wrote:
If you can follow a French tutorial, you can go on http://matthieu-brucher.developpez.com/tutoriels/python/swig-numpy/#LV to have a skeletton for your issue.
That looks very useful -- any chance of an English translation? My one year of high school French is proving useless. Otherwise, the code itself is still quite helpful. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
2008/10/30 Christopher Barker
Matthieu Brucher wrote:
If you can follow a French tutorial, you can go on http://matthieu-brucher.developpez.com/tutoriels/python/swig-numpy/#LV to have a skeletton for your issue.
That looks very useful -- any chance of an English translation? My one year of high school French is proving useless. Otherwise, the code itself is still quite helpful.
-Chris
I thought I put it on my blog, but no :| I may find one day the time to translate it and to improve/enhance it. Matthieu -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher
A few comments: Chris LeBlanc wrote:
I'm thinking I could create a NumPy array that uses the same C array (1d, 2d, or 3d) that our program is using instead of copying the memory.
yes, you can do that.
At this point, I'm thinking the python code wouldn't need to return any objects because it would be modifying the seismic data (and trace headers) in memory.
There are a number of operations in Python that would require copies of the data, and you may want to be able to create data in python, and process it with your C code, so you probably do want to support passing data arrays from Python to the C code as well.
Maybe calling a custom module from the Python code that does the C array to NumPy translation using Cython/pyrex/swig/etc. Would it be possible to use the same C arrays from here without copying them?
yes, and you probably do want to use one of Cython, SWIG, or Ctypes -- it's really easier not to have to handle the reference counting, etc, yourself. There are a set of SWIG typemaps and docs in the numpy distribution (in the docs dir, I believe), and there are various Wiki pages describing how to do it with Cython and ctypes as well. How to choose? My thoughts on SWIG: SWIG is a pretty complex system, so you need to learn what is essentially yet another language. However, the numpy.i typemaps work well for the simple cases, so it may be easy to get started. The big advantage of SWIG is that it auto-generates the wrapper, once you have defined the typemaps you need. This gives a real advantage if you need to : - provide wrappers for more than one language (PERL, Java, etc..) - are wrapping a substantial library, particularly one that is under active development. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
participants (3)
-
Chris LeBlanc -
Christopher Barker -
Matthieu Brucher