I have an old Numarray C extension (or, rather, a Python package containing a C extension) that I would like to convert to numpy (in a way that is likely to be supported long-term).
Options I have found include:
- Use the new numpy extension. This seems likely to be fast and future-proof. But I am finding the documentation slow going. Does anyone know of a simple example (e.g. read in an array, create a new array)?
- Use the Numarray compatible C API. Simple (and takes advantage of the nice Numarray tutorial example for documentation), but will this be supported in the long term?
- Switch to ctypes. Simple in concept. But I'm wondering if I can get distutils to build the resulting package.
- Use SWIG. I have some experience with it, but not with numpy arrays.
- Use Cython to replace the C code. No idea if this is a long-term supported package.
Another option is to try to rewrite in pure python. Perhaps the numpy indexing is sophisticated enough to allow an efficient solution. The C extension computes a radial profile from a 2-d masked array: radProf(r)= sum of all unmasked pixels at radius r about some specified center index I can easily generate (and cache) a 2-d array of radius index, but is it possible to use that to efficiently generate the desired sum?
Any opinions?
-- Russell
On Mon, Jun 29, 2009 at 3:03 PM, Russell E. Owen rowen@u.washington.eduwrote:
I have an old Numarray C extension (or, rather, a Python package containing a C extension) that I would like to convert to numpy (in a way that is likely to be supported long-term).
Options I have found include:
- Use the new numpy extension. This seems likely to be fast and
future-proof. But I am finding the documentation slow going. Does anyone know of a simple example (e.g. read in an array, create a new array)?
- Use the Numarray compatible C API. Simple (and takes advantage of the
nice Numarray tutorial example for documentation), but will this be supported in the long term?
- Switch to ctypes. Simple in concept. But I'm wondering if I can get
distutils to build the resulting package.
Use SWIG. I have some experience with it, but not with numpy arrays.
Use Cython to replace the C code. No idea if this is a long-term
supported package.
Another option is to try to rewrite in pure python. Perhaps the numpy indexing is sophisticated enough to allow an efficient solution. The C extension computes a radial profile from a 2-d masked array: radProf(r)= sum of all unmasked pixels at radius r about some specified center index I can easily generate (and cache) a 2-d array of radius index, but is it possible to use that to efficiently generate the desired sum?
Any opinions?
How big is the extension and what does it do?
Chuck
In article e06186140906291429m3cb339e8ge298f179d811e8a7@mail.gmail.com, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Jun 29, 2009 at 3:03 PM, Russell E. Owen rowen@u.washington.eduwrote:
I have an old Numarray C extension (or, rather, a Python package containing a C extension) that I would like to convert to numpy (in a way that is likely to be supported long-term).
Options I have found include:
- Use the new numpy extension. This seems likely to be fast and
future-proof. But I am finding the documentation slow going. Does anyone know of a simple example (e.g. read in an array, create a new array)?
- Use the Numarray compatible C API. Simple (and takes advantage of the
nice Numarray tutorial example for documentation), but will this be supported in the long term?
- Switch to ctypes. Simple in concept. But I'm wondering if I can get
distutils to build the resulting package.
Use SWIG. I have some experience with it, but not with numpy arrays.
Use Cython to replace the C code. No idea if this is a long-term
supported package.
Another option is to try to rewrite in pure python. Perhaps the numpy indexing is sophisticated enough to allow an efficient solution. The C extension computes a radial profile from a 2-d masked array: radProf(r)= sum of all unmasked pixels at radius r about some specified center index I can easily generate (and cache) a 2-d array of radius index, but is it possible to use that to efficiently generate the desired sum?
Any opinions?
How big is the extension and what does it do?
It basically contains 2 functions: 1: radProfile: given a masked image (2d array), a radius and a desired center: compute a new 1d array whose value at index r is the sum of all unmasked pixels at radius r.
2: radAsymm: given the same inputs as radProfile, return a (scalar) measure of radial asymmetry by computing the variance of unmasked pixels at each radius and combining the results.
The original source file is about 1000 lines long, of which 1/3 to 1/2 is the basic C code and the rest is Python wrapper.
-- Russell
On Mon, Jun 29, 2009 at 4:17 PM, Russell E. Owen rowen@u.washington.eduwrote:
In article e06186140906291429m3cb339e8ge298f179d811e8a7@mail.gmail.com, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Jun 29, 2009 at 3:03 PM, Russell E. Owen rowen@u.washington.eduwrote:
I have an old Numarray C extension (or, rather, a Python package containing a C extension) that I would like to convert to numpy (in a way that is likely to be supported long-term).
Options I have found include:
- Use the new numpy extension. This seems likely to be fast and
future-proof. But I am finding the documentation slow going. Does
anyone
know of a simple example (e.g. read in an array, create a new array)?
- Use the Numarray compatible C API. Simple (and takes advantage of the
nice Numarray tutorial example for documentation), but will this be supported in the long term?
- Switch to ctypes. Simple in concept. But I'm wondering if I can get
distutils to build the resulting package.
Use SWIG. I have some experience with it, but not with numpy arrays.
Use Cython to replace the C code. No idea if this is a long-term
supported package.
Another option is to try to rewrite in pure python. Perhaps the numpy indexing is sophisticated enough to allow an efficient solution. The C extension computes a radial profile from a 2-d masked array: radProf(r)= sum of all unmasked pixels at radius r about some specified center index I can easily generate (and cache) a 2-d array of radius index, but is
it
possible to use that to efficiently generate the desired sum?
Any opinions?
How big is the extension and what does it do?
It basically contains 2 functions: 1: radProfile: given a masked image (2d array), a radius and a desired center: compute a new 1d array whose value at index r is the sum of all unmasked pixels at radius r.
2: radAsymm: given the same inputs as radProfile, return a (scalar) measure of radial asymmetry by computing the variance of unmasked pixels at each radius and combining the results.
The original source file is about 1000 lines long, of which 1/3 to 1/2 is the basic C code and the rest is Python wrapper.
It sounds small enough that you should be able to update it to the numpy interface. What functions do you need? You should also be able to attach a copy (zipped) if it is small enough, which might help us help you.
Chuck
On Mon, Jun 29, 2009 at 6:10 PM, Charles R Harris <charlesr.harris@gmail.com
wrote:
On Mon, Jun 29, 2009 at 4:17 PM, Russell E. Owen rowen@u.washington.eduwrote:
In article e06186140906291429m3cb339e8ge298f179d811e8a7@mail.gmail.com, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Jun 29, 2009 at 3:03 PM, Russell E. Owen rowen@u.washington.eduwrote:
I have an old Numarray C extension (or, rather, a Python package containing a C extension) that I would like to convert to numpy (in a way that is likely to be supported long-term).
Options I have found include:
- Use the new numpy extension. This seems likely to be fast and
future-proof. But I am finding the documentation slow going. Does
anyone
know of a simple example (e.g. read in an array, create a new array)?
- Use the Numarray compatible C API. Simple (and takes advantage of
the
nice Numarray tutorial example for documentation), but will this be supported in the long term?
- Switch to ctypes. Simple in concept. But I'm wondering if I can get
distutils to build the resulting package.
Use SWIG. I have some experience with it, but not with numpy arrays.
Use Cython to replace the C code. No idea if this is a long-term
supported package.
Another option is to try to rewrite in pure python. Perhaps the numpy indexing is sophisticated enough to allow an efficient solution. The C extension computes a radial profile from a 2-d masked array: radProf(r)= sum of all unmasked pixels at radius r about some specified center index I can easily generate (and cache) a 2-d array of radius index, but is
it
possible to use that to efficiently generate the desired sum?
Any opinions?
How big is the extension and what does it do?
It basically contains 2 functions: 1: radProfile: given a masked image (2d array), a radius and a desired center: compute a new 1d array whose value at index r is the sum of all unmasked pixels at radius r.
2: radAsymm: given the same inputs as radProfile, return a (scalar) measure of radial asymmetry by computing the variance of unmasked pixels at each radius and combining the results.
The original source file is about 1000 lines long, of which 1/3 to 1/2 is the basic C code and the rest is Python wrapper.
It sounds small enough that you should be able to update it to the numpy interface. What functions do you need? You should also be able to attach a copy (zipped) if it is small enough, which might help us help you.
And yes, you could probably do it in cython and save yourself a bit of interface code. I think cython will currently handle 2d arrays without much trouble. But I don't know what the fastest approach would be here.
Chuck
In article e06186140906291710s34865590p38032012f12d0f60@mail.gmail.com, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Jun 29, 2009 at 4:17 PM, Russell E. Owen rowen@u.washington.eduwrote:
In article e06186140906291429m3cb339e8ge298f179d811e8a7@mail.gmail.com, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Jun 29, 2009 at 3:03 PM, Russell E. Owen rowen@u.washington.eduwrote:
I have an old Numarray C extension (or, rather, a Python package containing a C extension) that I would like to convert to numpy (in a way that is likely to be supported long-term).
How big is the extension and what does it do?
It basically contains 2 functions: 1: radProfile: given a masked image (2d array), a radius and a desired center: compute a new 1d array whose value at index r is the sum of all unmasked pixels at radius r.
2: radAsymm: given the same inputs as radProfile, return a (scalar) measure of radial asymmetry by computing the variance of unmasked pixels at each radius and combining the results.
The original source file is about 1000 lines long, of which 1/3 to 1/2 is the basic C code and the rest is Python wrapper.
It sounds small enough that you should be able to update it to the numpy interface. What functions do you need? You should also be able to attach a copy (zipped) if it is small enough, which might help us help you.
It is the PyGuide package http://www.astro.washington.edu/rowen/PyGuide/files/PyGuide.zip a 525k zip file. The extension code is in the src directory.
I would certainly be grateful for any pointers to how the old numarray C API functions map to the new numpy ones. I would prefer to use the new numpy API if I can figure out what to do.
-- Russell
On Tue, Jun 30, 2009 at 12:31 PM, Russell E. Owen rowen@u.washington.eduwrote:
In article e06186140906291710s34865590p38032012f12d0f60@mail.gmail.com, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Jun 29, 2009 at 4:17 PM, Russell E. Owen rowen@u.washington.eduwrote:
In article e06186140906291429m3cb339e8ge298f179d811e8a7@mail.gmail.com, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Jun 29, 2009 at 3:03 PM, Russell E. Owen rowen@u.washington.eduwrote:
I have an old Numarray C extension (or, rather, a Python package containing a C extension) that I would like to convert to numpy (in a way that is likely to be supported long-term).
How big is the extension and what does it do?
It basically contains 2 functions: 1: radProfile: given a masked image (2d array), a radius and a desired center: compute a new 1d array whose value at index r is the sum of all unmasked pixels at radius r.
2: radAsymm: given the same inputs as radProfile, return a (scalar) measure of radial asymmetry by computing the variance of unmasked
pixels
at each radius and combining the results.
The original source file is about 1000 lines long, of which 1/3 to 1/2 is the basic C code and the rest is Python wrapper.
It sounds small enough that you should be able to update it to the numpy interface. What functions do you need? You should also be able to attach
a
copy (zipped) if it is small enough, which might help us help you.
It is the PyGuide package http://www.astro.washington.edu/rowen/PyGuide/files/PyGuide.zip a 525k zip file. The extension code is in the src directory.
I would certainly be grateful for any pointers to how the old numarray C API functions map to the new numpy ones. I would prefer to use the new numpy API if I can figure out what to do.
You can look at the numpy/numarray/_capi.c file where the translation from numpy to numarray is located. A lot of the functions map directly, others are more complicated.
Chuck
On Tue, Jun 30, 2009 at 12:31 PM, Russell E. Owen rowen@u.washington.eduwrote:
In article e06186140906291710s34865590p38032012f12d0f60@mail.gmail.com, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Jun 29, 2009 at 4:17 PM, Russell E. Owen rowen@u.washington.eduwrote:
In article e06186140906291429m3cb339e8ge298f179d811e8a7@mail.gmail.com, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Jun 29, 2009 at 3:03 PM, Russell E. Owen rowen@u.washington.eduwrote:
I have an old Numarray C extension (or, rather, a Python package containing a C extension) that I would like to convert to numpy (in a way that is likely to be supported long-term).
How big is the extension and what does it do?
It basically contains 2 functions: 1: radProfile: given a masked image (2d array), a radius and a desired center: compute a new 1d array whose value at index r is the sum of all unmasked pixels at radius r.
2: radAsymm: given the same inputs as radProfile, return a (scalar) measure of radial asymmetry by computing the variance of unmasked
pixels
at each radius and combining the results.
The original source file is about 1000 lines long, of which 1/3 to 1/2 is the basic C code and the rest is Python wrapper.
It sounds small enough that you should be able to update it to the numpy interface. What functions do you need? You should also be able to attach
a
copy (zipped) if it is small enough, which might help us help you.
It is the PyGuide package http://www.astro.washington.edu/rowen/PyGuide/files/PyGuide.zip a 525k zip file. The extension code is in the src directory.
I would certainly be grateful for any pointers to how the old numarray C API functions map to the new numpy ones. I would prefer to use the new numpy API if I can figure out what to do.
This doesn't look too bad, I only count 5 functions/macros.
NA_InputArray NA_OutputArray NA_ShapeEqual NA_NewArray NA_OFFSETDATA
The quick and dirty solution would be to just copy those functions in at the top of your code. You might want to fix up the NumarrayType enum instead of including it and a few other such.
The code looks like it would go over into cython fairly nicely since it is split between interface code, which would look good in python, and a couple of pure c functions. If you have the time that might be a good way to go.
Chuck
Hi Russell,
Have you looked at the example in our interactive data analysis tutorial where we compute radial profiles in Python? It's not as fast as C because of the sort, but perhaps that's fast enough for your purposes. I wasn't sure if you had already seen that approach or not. (I think it is in the 3rd chapter, but I can look it up if you need me to).
Perry
On Jun 29, 2009, at 5:03 PM, Russell E. Owen wrote:
I have an old Numarray C extension (or, rather, a Python package containing a C extension) that I would like to convert to numpy (in a way that is likely to be supported long-term).
Options I have found include:
- Use the new numpy extension. This seems likely to be fast and
future-proof. But I am finding the documentation slow going. Does anyone know of a simple example (e.g. read in an array, create a new array)?
- Use the Numarray compatible C API. Simple (and takes advantage of
the nice Numarray tutorial example for documentation), but will this be supported in the long term?
- Switch to ctypes. Simple in concept. But I'm wondering if I can get
distutils to build the resulting package.
Use SWIG. I have some experience with it, but not with numpy arrays.
Use Cython to replace the C code. No idea if this is a long-term
supported package.
Another option is to try to rewrite in pure python. Perhaps the numpy indexing is sophisticated enough to allow an efficient solution. The C extension computes a radial profile from a 2-d masked array: radProf(r)= sum of all unmasked pixels at radius r about some specified center index I can easily generate (and cache) a 2-d array of radius index, but is it possible to use that to efficiently generate the desired sum?
Any opinions?
-- Russell
Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
In article 4D2B04ED-4612-4244-A8B8-3FF0C8659582@stsci.edu, Perry Greenfield perry@stsci.edu wrote:
Hi Russell,
Have you looked at the example in our interactive data analysis tutorial where we compute radial profiles in Python? It's not as fast as C because of the sort, but perhaps that's fast enough for your purposes. I wasn't sure if you had already seen that approach or not. (I think it is in the 3rd chapter, but I can look it up if you need me to).
I have not seen this. I'll give it a look. Thanks! But I suspect the sort will add unacceptable overhead because this radial profile is computed as part of an iteration (to find the point of maximum radial symmetry).
-- Russell